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ABSTRACT 

In the first section of this book, seven papers are 
presented which were given at the working conference on Computer 
Applications in Dental Education held in San Francisco in October 
1969. Three of the papers are statements by educators (Lawrence M. 
Stolurow, Ralph E. Grubb, and John A. Starkweather) who have been 
active for several years in the area of computer-assisted instruction 
(CAI) outside the context of dental education. The remaining papers 
deal with the use of computers in four areas of dental education: 
admissions, instructions, dental school operations, and evaluation of 
clinical course work. In the second section, along with brief reports 
on the four conference discussion groups, summaries are presented of 
computer projects in progress at the dental schools represented at 
the conference. The project summaries are divided into four areas of 
application: Admissions and academic evaluation, CAI, dental school 
operations and educational administration, and student evaluation in 
clinical course work. The types of projects summarized include 
computer assistance in the admission process, automated scanning 
analysis of dental X-rays, development of an automated dental 
training simulator, and computerized student evaluation systems. A 
directory of conference participants includes a brief indication of 
the computer system, areas of application, and project status of each 
conference. (Author/JY) 
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DISCRIMINATION PROHIBITED - Title VI of the Civil 
Rights Act of 1964 states: “No person in the United States 
shall, on the ground of race, color or national origin, be 
excluded from participation in, be denied the benefits of, or 
be subjected to discrimination under any program or activity 
receiving Federal financial assistance. Therefore, programs 
in dental education, like every program or activity receiving 
Federal assistance from the Department of Health, Education, 
and Welfare, must be operated in compliance with this law. 
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PREFACE 



In recent years our nation’s dental schools have shown a 
growing interest in using computers to solve their educational 
problems. In order to promote cooperation and communication 
among dental schools in their efforts, a working conference on 
" Computer Applications in Dental Education” was held at San 
Francisco, California during October 29-31, 1969. Sponsored by 
the Professional Education Branch (of the Division of Dental 
Health, Bureau of Health Manpower Education, National Insti- 
tutes of Health), the meeting was attended by some eighty parti- 
cipants representing thirty-five American and Canadian dental 
schools, by staff members of the Division of Dental Health, and 
by observers from the National Library of Medicine and the 
Division of Physician Manpower, BHME, NIH. 

In addition to bringing together dental educators to exchange 
information and discuss present problems, the conference was 
intended as a means of providing guidance for dental schools 
which in the future may come to consider the use of computers. 
Consequently, this report has been published to provide a readily 
available source of information as more dental schools begin to 
introduce computers into some phase of the educational process. 

Part 1 consists of seven papers based on formal presenta- 
tions made by the invited speakers. Part 2 contains summaries 
of computer projects in progress at the dental schools represented 
at the conference, organized by four areas of application: (1) ad- 
missions and academic evaluation, (2) computer-assisted instruc- 
tion, (3) dental school operations and educational administration, 
and (4) student evaluation in clinical coursework. The appendix 
lists the conference participants with their addresses and tele- 
phone numbers as of May 197 0 ^ with each entry followed by a 
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brief indication of the computer system the individual is using, 
his area of application, and the status of his project. 

The publications from three recent meetings contain infor- 
mation of interest to the dental educator who is beginning to 
explore the uses of computers. In February 1968, a Conference 
on Computer-Assisted Instruction in Medical Education was held 
at the Harvard University Medical School. The proceeds of the 
meeting, entitled Computer Assisted Instruction in the Health 
Professions (Stolurow, Peterson & Cunningham, eds., 1970) can 
be obtained from ENTELEK, INC., Newburyport, Massachusetts 
01950. In April 1968, the Conference on the Use of Computers 
in Medical Education was held at the University of Oklahoma, 
School of Medicine. The Proceedings of that conference are avail- 
able from the Office of Information, Division of Physician and 
Health Professions Education, BHME, NIH, 9000 Rockville Pike, 
Bethesda, Maryland 20014. In June 1969, a conference on Com- 
prehensive Care in Clinical Dental Education was sponsored by 
the University of DJprth Carol i an and copies of the Proceedings 
are available from Dr. Clifton E. Crandell, University of North 
Carolina, School of Dentistry, Chapel Hill, North Carolina 
27515. In addition, the Index to Computer Assisted Instruction 
(Lekan, ed., 1970) available through the Sterling Institute, 3750 
Prudential Tower, Boston, Massachusetts 02199, summarizes 
some eighty computer-assisted instruction programs written for 
health professions education. 

I would like to thank the authors of the papers and project 
summaries for reviewing their material for publication and to 
acknowledge the labor of the editors who transformed the manu- 
scripts into a book: Luigi F. Lucaccini, Martin L. MacIntyre 
and Jack Handley. 



Dale W. Podshadley, D.D.S. 
Conference Chairman 



i 



i 

■! 



11 



DALEW PODSHADLEY is Chief, Professional Education Branch, Division of Den- 
tal Health, Bureau of Health Professions Education and Manpower Training, National 
Institutes of Health, San Francisco, California. He received his D D S from St 
Louis University and his M.P.H. from the University of Michigan. He has directed 
an ongoing program of applied research in dental education for the Division of 
Dental Health sinee 1961. Present program areas include the recruitment, selection, 
and development of dental students and teachers; the application of modern instruc- 
tional technology to dental education; the development of continuing dental education 
systems; human faetors in the design of dental instructional facilities; and the ad- 
ministration^ of dental educational systems. In addition to having published numerous 
papers in the area of dental education, he is eo-author of the programmed text, 
Introduction to Dental Public Health. He serves on the faculties of the University 
of California Sehool of Dentistry and the University of Pacific School of Dentistry, 
and is a Diplomate of the American Board of Dental Public Health. 



INTRODUCTION 



Dale W. Podshadley 

Professional Education Branch, Division of 
Dental Health 



One of the most significant events of recent years was the 
public acknowledgment by the health professions that health is a 
right of the people. When we in the health fields acknowledged 
this, we immediately accepted the responsiblity to see that care 
is available to all. Given the shortage of health manpower, this 
responsibility constitutes a tremendous challenge. It is obvious 
that the health professions must develop new systems for deliver- 
ing comprehensive health services. The resources of personnel, 
administrative mechanisms, and facilities must be organized in 
new subsystems to radically increase their efficiency, effectiveness 
and economy before the public can receive the care it needs. 

Let us look briefly at some of the subsystems which com- 
prise dental education; in particular, students, faculty, adminis- 
tration, facilities, and practicing dentists. In order to increase 
manpower, new schools have been created and existing ones 
have expanded their enrollments. Between 1965 and 1975 there 
will be more than a one-third increase in the number of grad- 
uates, finally reaching 4,300 annually. Unfortunately, this dra- 
matic and necessary increase in the number of students has 
compounded the problem of the shortage of teachers. As Dr. 
Diefenbach emphasized in his remarks: 
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Our dental schools find themselves hard-pressed to 
retain qualified teaching staffs. Those people who do 
accept the challenge of teaching find it increasingly 
difficult to keep up to date in dental science. Increas- 
ing class sizes, the need for continuous curriculum 
improvement, and the need to individualize instruc- 
tion are some of the factors which work against 
teachers operating at optimal effectiveness and effi- 
ciency.* . i 

The expansion of the dental curriculum deserves special 

recognition. Because of social change and increased social aware- 
ness, many new content areas are being added which previously 
were not considered as being essential in the practice of dentistry. 
Elements of psychology, sociology, anthropology, economics, and 
political science are but a few of the disciplines now being in- 
cluded in dental education. At the same time the knowledge 
explosion has greatly added to the content of traditional courses 

and will continue to do so. 

These factors make it imperative that new and im- 
proved educational and administrative methods be 
sought out and applied as rapidly as possible. VI e 
must ensure the production of graduates in the quan- 
tity and of the quality required to maintain and im- 
prove the oral health of the American people. The 
time is now for dental educators to study carefully 
the potential of computer technology and to begin to 
apply it effectively to their problems on a large scale 
basis ** 

Let us look now at problem areas where computer technology 
can provide solutions. In the manpower area a great deal has 
been written about the expanded use of auxiliaries, including the 
development of entirely new disciplines, as a partial answer to 
the manpower shortage. Very little, however, has been written 
about the problems of recruiting and selecting individuals in a 
manner to ensure career satisfaction. There is, probably, little 
economy in expanding the duties of auxiliaries if attrition con- 
tinues at the present rate. Almost the same problem exists in 
regard to training dentists. In dental practice of the future, it is 
likely that the dentist will be a specialist in diagnosis and treat- 
ment planning, community leadership, and administration. This 



* Welcoming address of Dr. Viron L. Diefenbach, former director of the Division of 
Dental Health, Bureau of Health Professions Education and Manpower Training, 
National Institutes of Health, Public Health Sendee, U.S. Department of Health, 
Education, and Welfare. 

** Diefenbach. op. cit. 
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new type of practice will require a person different from the one 
who is now choosing a career in dentistry. 

Data banks are one answer to both of these problems of 
selecting people for dental careers. Computers are the only answer 
to the management of data banks. With cumulative information, 
it should be possible to continually update the selection criteria 
to meet the needs of society. A great deal of research and de- 
velopment effort is needed, particularly in identifying the factors 
which determine ff success” in private practice. 

Assuming that students can be selected on the bases of 
career success and happiness, it is important, next, to ensure 
that each student be allowed to realize his potential. I doubt 
very much if the students of the future are going to settle for 
less. They are going to expect freedom and the right to exercise 
their individual initiative. Probably of greatest need is curricular 
flexibility to meet the interest and capabilities of individual stu- 
dents. Although some form of core curriculum might be required, 
beyond that point students must be given maximum freedom if 
they are to develop fully. 

In recent years there has been a great deal of interest in the 
processes of teaching and learning. For the problems of the 
teacher shortage and of reaching a large number of students, 
educational television was somewhat of an answer. To provide 
information when a student wants it, dial and random access 
capabilities have been developed. To enrich learning experiences 
by involving more than one sense mode, single- and multiple- 
concept films and filmstrips are now available. 

To personalize and individualize instruction, utilizing the 
learning principles of immeidate reinforcement and active parti- 
cipation, one answer is computer- assisted instruction (CAI), 
Only CAI would appear at this point to answer the problems 
arising from individual goals. I doubt if adequate curriculum 
flexibility can be achieved without it. Developing people to their 
potential implies continuing their education until they reach their 
peak. Practically speaking, this suggests that a girl who originally 
enrolls in a dental assisting curriculum should be able to go on 
to become a dentist or pediatrician or nuclear physicist, if she 
desires, with minimum repetition of coursework and loss of time. 

An automated system with almost unlimited storage, retrieval 
and evaluation capabilities is essential. A retrieval system under 
computer control merits some elaboration. A library of prerecord- 
ed instructional materials immediately accessible to the learner 
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when he needs them would provide the required flexibility and 
individualization. This is of major importance in the teaching of 
psychomotor skills, where students differ so greatly in their con- 
ceptaul and digital abilities. The student could proceed on his own 
in accord with his own capabilities. Since information can be 
dispensed electronically, there will be more dialogue and com- 
munication between teacher and learner, and more time can be 
devoted to individual learning problems. 

Computer-assisted instruction, however, is only one of a 
number of computer applications that can improve dental educa- 
tion. Many of you are involved in using computers to automate 
student progress records and to continually evaluate students’ 
clinical performance. After refinement of this information, it 
should be possible to develop learning curves in the motor skills 
area. Such curves are a fundamental requirement if we are 
eventually to let a student proceed independently and yet pre- 
cisely measure his learning. Many of you also are involved in 
individual test analyses; why not also attempt to establish a 
central data bank of test items which could be available to 
faculty throughout the country? This would have a significant 
benefit: better and certainly more objective tests, with consider- 
ably less time and effort required from overburdened teachers. 
There are many other computer uses which can directly improve 
instruction. For example, learning environments can be simu- 
lated with controlled variables for the study of learner-instructor 
interaction. 

Let us assume now that a student has been selected for 
dental school, has had the freedom to complete the curriculum 
in two years and is happy and successful in private practice. 
The computer will still have a role: to ensure that this dentist 
is provided with the information needed to keep him up to 
date. It should be possible for his educational needs to be con- 
tinually diagnosed and treated, possibly through strategically 
located computers, on the basis of his type of practice, his loca- 
tion, and the specific characteristics of the population he serves. 
It is likely, because of the knowledge explosion, that continuing 
education will have to be managed in this way, and with this 
degree of selectivity, to maintain the skills of practicing dentists. 

The implication of computers for dental education are in- 
deed far-reaching. Billing, filing, payroll, inventory control, 
and a host of routine problems can now be handled more effi- 
ciently by computers. These functions, although only indirectly 

15 
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related to the instructional process, have a distinct influence on 
the educational economy. 

One computer application which emphasizes the interrela- 
tionships between students, teachers, facilities, and the public 
deserves special consideration. This is, as I mentioned' earlier, 
its use to improve the deliver)- of care, specificallv within an 
educational setting. If dental schools are to continue attracting 
patients from the general population, then a great deal must be 
done to increase the efficiency with which these patients receive 
treatment. But this must also be done in a way that considers 
the students and learning first. Part of the problem is in 
scheduling; much can be done to implement comprehensive pa- 
tient care, and, at the same time, make optimal use of the 
student's time, the faculty's time, the patient's time, and the 
facilities. The application of operations research and systems 
analysis techniques to improve the efficiency of clinical opera- 
tions is severely needed. 

To conclude, there are two major purposes of this Con- 
ference. First, to stimulate communication and encourage cooper- 
ation among the schools now using computers. Second, to pro- 
vide some documentation which may assist other schools who 
are contemplating the use of computers. If these aims are 
achieved, at least in part, then the Conference will have been 
worthwhile. The future holds exciting possibilities if we build 
progressively on one another s knowledge and experience. 



PART 1 



Invited 

Addresses 



o 

ERIC 



17 



i ' ~ " ^ ; 

| Parti. INVITED ADDRESSES 

I 

1 INTRODUCTION 




This section contains seven papers based on formal adddres- 
ses made at the conference. The first three (Stolurow, Grubb, 
and Starkweather) are statements by educators who have been 
active for several years in the area of computer- assisted instruc- 
tion (GAI), primarily outside the context of dental education. The 
remaining papers deal with the use of computers in four areas 
of dental education— admissions (Ginley), CAI (Tira), dental 
school operations (Crandell), and evaluation of clinical course- 
work (Ehrlich). 

Lawrence M. Stolurow prefaces his description of CAI pro- 
grams and projects at the Harvard CAI Laboratory with a dis- 
cussion ol educational technology and the modes of interactive 
instruction available in CAI. Stolurow is interested both in pro- 
viding interactive instruction and in using CAI as a vehicle for 
conducting basic studies in learning. He illustrates possible in- 
structional strategies with samples of student-computer dialogue 
taken from CAI programs in medical and dental education at 
Harvard. 

Ralph E. Grubb of IBM characterizes CAI as the use of a 
computer to improve the student’s interaction with materials, 
subject matter, and teachers, with stress placed on the nature of 
the interactive process. Eleven modes of interaction are identified, 
ranging from drill and practice to exploration. He notes that 
since we are unable to individualize instruction successfully in a 
prescriptive fashion at present, a more fruitful approach, at least 
on an interim basis, might be to relinquish program control to 
the learner to make selections of experiences within a learning 
space or content area according to his own understanding of his 
needs. Grubb enters a plea for serious consideration of the 
exploratory or "learner-controlled” mode, citing studies which 
demonstrate its advantages. He also reviews briefly the resources 
needed in a successful CAI effort and’ sketches some avenues 
that future CAI work may take. 

John A. Starkweather of the University of California, San 
Francisco Medical Center, describes thfe operation of a CAI sys- 
tem (PILOT) within a multipurpose computing center. Although 

18 
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the diverse demands of the user group make it difficult to con- 
centrate resources on any one aspect of the services offered, 
PILOT, actually an interactive, conversational programming 
language, has enjoyed considerable success. PILOT affords a 
variety of instructional modes, including simulation, through its 
natural language processing capabilities. Descriptions of patient 
simulation programs which are intended for use in teaching 
medical interviewing skills are included, as well as examples of 
programs which illustrate some of the other instructional modes 
possible with PILOT. Starkweather outlines several areas of 
need, emphasizing in particular the desirability of translating 
curriculum elements from one CAI system to another. Compati- 
bility of this sort would have several advantages, including 
multiple evaluation of new materials, joint curriculum develop- 
ment, and the sharing of existing materials. 

Thomas J. Ginley of the American Dental Association con- 
siders the growing problem faced by dental schools in processing 
applicants for admissions. An increase of nearly one-half in the 
number of candidates is expected over the next five years, ac- 
companied by a corresponding increase in the number of docu- 
ments that must be handled by admissions committees. To 
reduce the burden now placed on the individual dental school 
in processing applicant records and in selecting an entering 
class, Ginley proposes three alternatives, ranging from a docu- 
ment clearinghouse to a central computerized selection system. 
His paper makes clear the scope of the problem and the need 
for developing a procedure superior to the one now in use. 
Whatever form such a system might take, it is clear that the 
cooperation of all dental schools is necessary if it is to be 
successful. 

Daniel E. Tira describes experiences with CAI courses in 
physiological chemistry (14 hours in length) and oral cancer 
recognition (4 hours) at the Ohio State University, College of 
Dentistry. The first course features a tutorial, self-evaluative 
format which has gained acceptance by students in spite of the 
novelty of format and the severe scheduling problems encounter- 
ed. The second is an attempt to simulate the decision-making 
processes which might occur in practice in the diagnosis of an 
oral lesion. The program permits the student to develop a pa- 
tient history, conduct a physical examination, and request lab- 
oratory tests in an unstructured fashion. The student’s differen- 
tial diagnosis is evaluated and either confirmed or rejected with 
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relevant feedback until the correct diagnosis is made. The initial 
resistance of dental students to these courses bears witness to 
the problems faced by a lone self- instructional course when it is 
injected into a curriculum in which self-evaluation and flexible 
scheduling are traditionally minimized. On the positive side, the 
project demonstrates that CAI, often narrowly viewed as a means 
of supplementing or supplanting classroom instruction, can 
provide another and equally valuable benefit, that of functioning 
as an impersonal and unflagging evaluation system. In this 
project the CAI system provides (1) self-evaluation to the stu- 
dent, (2) information for faculty upon which to base tutoring 
and counseling, and (3) a source for subject matter validation 
for the curriculum specialist. 

Clifton E. Crandell reports on the work in progress at the 
University of North Carolina, School of Dentistry to introduce 
total patient care into the clinic experiences of dental students. 
Until now, the requirement of developing technical skill has been 
incompatible with comprehensive patient care because of the com- 
plexity of scheduling student clinic experiences within such con- 
straints as the needs of the patient, the availability of faculty 
specialists and facilities, and the level of ability of the individual 
student. Computer-based operations research methods appear to 
offer one solution to the scheduling problem posed by compre- 
hensive patient care. A scheduling model based on linear pro- 
gramming techniques has been used with a limited number of 
students and preselected patients in a pilot study. Some prelim- 
inary results are discussed along with plans to extend the 
approach to include all students and clinic patients at North 
Carolina. 

Paul Ehrlich describes the recordkeeping system in opera- 
tion at the University of Southern California, School of Dentistry 
for evaluation of students’ clinical performance. Measures of 
performance include grades, the quantity of work performed, 
and a composite measure based on both quality and quantity. 
Among the advantages of the computerized system are the rapid 
provision of progress reports to students and faculty, the auto- 
matic compilation of grades, and increased revenue from clinic 
operations as a by-product of tighter controls. Ehrlich discusses 
some of the practical problems which occur in implementing 
and operating a system of this type. 
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LEARNING IN THE MEDICAL/DENTAL 
SCHOOL ENVIRONMENT 



Lawrence M. Stolurow 
Harvard University 



I am pleased to have this opportunity to share with you 
some thoughts about learning in the medical and dental school 
environment. 

Dental and medical educational programs, particularly at the 
undergraduate level, are currently undergoing extensive review 
and examination. As Dr. Podshadley has indicated, the need for 
change is patently clear. He has mentioned several factors con- 
tributing to the urgency of the situation. The three most critical 
from my point of. view are: the ' 'information explosion” in the 
basic sciences; the need for increased services to meet population 
increases; and student insistence upon more relevant and mean- 
ingful school experiences. The problem for the schools is to 
respond to these requirements for change in an effective and 
meaningful way while at the same time maintaining academic 
excellence. 

Dentistry and medicine are no strangers to change. There 
has always been the problem in these two disciplines of accom- 
modating new developments and concepts and translating new 



The work reported in tins paper was conducted with the support of Office of Naval 
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tract No. PH 108-69-18. 



21 



LEARNING IN THE DENTAL SCHOOL ENVIRONMENT 13 

findings and theories into practical applications. The need to do 
these things is not new. What is new today is the tempo and 
magnitude of the changes. Both have increased several-fold over 
the past few decades. In fact, it is currently estimated that new 
knowledge in medicine is increasing at a rate of nine percent 
per year, and the rate is increasing. Dr. Podshadley’s projections 
suggest the rate ol acceleration we can expect. It is these dimen- 
sions of the present problem that are different from the past. It 
is these factors that are causing the difficulty. 

One approach to the solution of these problems is to turn 
to technology to accelerate what is now being done. Another is 
to examine the problem in the light of new concepts and 
methods of instruction to determine whether a new approach 
might be taken. The depth and extent of the problems is such 
that the mere use of isolated devices such as films, television, 
and tapes is not likely to solve them. The nature and complexity 
of the problems suggest that a more useful approach would be 
a continuing program of research that utilizes an integrated 
instructional system rather than a collection of devices. It is 
quite clear that both a conceptual and a research approach are 
needed to solve the problems, but that the institutional mecha- 
nism to bring it about in a systematic way is only just emerg- 
ing. 

There is also an attitudinal obstacle. Many faculty members 
do not consider it necessary to conduct in vivo studies of educa- 
tional problems. Moreover, not all health scientists feel that the 
problems of instruction are amenable to a scientific approach. 
Experience, however, suggests that this is just what is required 
if change is to mean progress. Research is as much needed for 
the solution of educational problems as for the solution of sub- 
stantive ones in the sciences. Educational problems must be 
identified, hypotheses formulated regarding their solution, and 
a plan developed and carried out to determine the answer. It 
can also be argued that the students who participate in educa- 
tional studies are taking a smaller risk than those who do not 
if one accepts the possibility that students taught by methods 
presently in use are not necessarily being taught by the best 
methods. Participation in educational studies can be immediately 
advantageous to the student, and the results will help indicate 
the nature of the changes that will make instruction better for 
the next group. Consequently, the profession and the patients 
both benefit from research and development in instruction. 



14 LEARNING IN THE DENTAL SCHOOL ENVIRONMENT 



The idea of experimentally testing ideas about instruction is 
winning increased acceptance. However, the scientific study of 
the instructional problems in dentistry and other health areas 
requires a laboratory, professional personnel, and a program. A 
team of specialists from several professional fields is needed so 
that scientific methods can be used to study the problems of 
learning and technology can serve the process. The concepts, 
methods, and tools for this approach are becoming available, 
but they must be developed within an educational context, i.e., 
there must be a symbiotic relationship between the host field 
and educational technology. 

EDUCATIONAL TECHNOLOGY 

Many think of educational technology as simply the use of 
audio or visual devices in instruction. While the application of 
educational technology does involve hardware, including com- 
puters, the use of these devices does not adequately describe the 
field. Media are one ingredient of a systematic educational tech- 
nology. Another, and critical, ingredient is psychology. The 
concepts and principles used to determine the organization of a 
course and the teaching strategies to be used are of greater 
potential value than the devices that implement them. The third 
ingredient of educational techology is research— the use of 
experimental methodology for decision making, quality control, 
and program improvement. Only research provides the experi- 
mental data which can be used to improve the instructional 
system and its use. 

A critical element of any learning system is the applied 
psychology of learning, retention, and transfer. As Dr. Pod- 
shadley mentioned,' dentistry is concerned with perceptual-motor 
learning as well as cognitive psychology, problem solving, rea- 
soning, and thinking. Concepts from these areas provide guidance 
for the design of instructional materials. The areas of testing and 
evaluation, including the measurement of aptitudes and person- 
ality, provide a rational basis for assessment and for carrying 
out specific types of instruction for individual students. 

The process of applying psychology to instruction has defi- 
nite steps. It begins with the definition of certain behavioral 
objectives for the instructional program. This means that state- 
ments are formulated to describe what a student will be able to 
do when he has successfully completed the course of instruction 
being prepared. Behavioral objectives are specific statements. They 
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are not general descriptions of the course and its philosophy. 

To ensure that the learner gets only what he needs during 
learnings, his instruction begins with a pretest. The pretest 
determines the amount of knowledge or skill a student has that 
is relevant to the instruction he is about to receive. After in- 
struction, the student is given a post-test, which provides the 
teacher or system with the information needed to determine the 
amount of learning that has taken place. The primary measure 
of success or failure of the instructional system is derived from 
the differences in pre- and post-test scores. 

Although these tests are essentially an achievement test, 
they are not constructed in the conventional way. Unlike most 
achievement tests, they are criterion-referenced . This is another 
way of saying that the items which make up the tests are in a 
one-to-one correspondence with the list of instructional objectives. 
The test items cover all of the things which the program is 
designed to teach. Failure on an item pinpoints what the student 
did not learn. 

Another critical part of the plan for an instructional system 
is its instructional strategy. This is the means by which the 
organization of the instructional material, including the way in 
which it relates to the learner, is achieved. An instructional 
strategy is a set of rules that determines the way in which a 
source of information, such as a teacher or instructional program, 
responds to the student. Even noninteractive forms of instruction 
such as lectures, films, and television, , vary in the way the 
information is organized for presentation to the student. How- 
ever, a strategy for interactive instruction is much more compli- 
cated; several methods of organization are planned so that the 
most appropriate one can be used for each student, depending 
upon his pattern of responses. 

In interactive instruction, information may be presented in a 
variety of ways. For example, it may be presented in an induc- 
tive manner or in a deductive manner. Also, each of these forms 
of presentation can be prepared so that the speed of the interac- 
tion is more or less rapid and proceeds in less or greater depth, 
depending upon the student’s performance. A sequence of instruc- 
tional units designed to teach a set of objectives is organized 
differently whenever different instructional strategies are used. 
For example, elements of a strategy could be the use of review 
procedures when a student makes a particular type of error, the 
use of review following a discussion of a set of topics, the use 
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of a correction procedure when errors are made, and the use of 
evaluative feedback. The term strategy is used here to mean the 
method of organizing instructional experiences, and the manner 
of relating information to the student. The means of achieving 
any organization of experiences is usually a set of rules that 
guide the interaction. The rules are stated in a contingency 
form, as "if . . . then . . . ” statements. A number of rules may 
be used together to provide a particular strategy. One set may 
determine the sequence, another the type of feedback, another 
the wording of the presentation. Usually more than one rule is 
used in structuring an interactive strategy. In principle, however, 
the concept of a strategy does not require more than one rule. 

Instruction is often said to have a "style”. This term gen- 
erally includes characteristics of the instructor’s behavior such 
as his personality and manner of relating to students. Style 
does not just refer to the rules of presentation. From the point 
of view of the educational technologist, the rules for selecting 
examples, the rules for sequencing topics, and the rules for 
evaluating student performance are the kinds of rules generally 
included in the description of a strategy. 

The dictionary definition of the term strategy refers to 
military planning of an interaction with an enemy. There is no 
attempt here to use the term in that sense. Rather, it is used 
in educational technology in the sense it is used in game theory , 
as a set of rules that determine the nature of an re-person inter- 
action. These rules are explicit and often formalized statements. 
The coding of computer-aided instruction (CAI) programs defines 
the nature of the interaction and is designed to produce a payoff 
in student performance. 

INDIVIDUALIZED INSTRUCTION 

While it has not been possible in the past to conceive of 
individualized instruction on a large scale, the introduction of 
the computer as a multiple-access, interactive device has made it 
possible to demonstrate that this approach is both feasible and 
practicable. 

It is important to define the meaning of individualized 
instruction as it is used here, before describing the various 
modes of instructional interaction. Individualized instruction is 
instruction that is prescriptive. The instructional interactions in 
which an individual engages are. selected (prescribed) on the 
basis of his performance. This prescription is based upon the 
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most relevant data available to the decision maker. The decisions 
to be made relate to both the content and the strategy of the 
instruction. Individualized instruction does not refer to instruction 
in isolation, which is its usual connotation. The prescription 
given a student may include a period of individual instruction 
at a CAI console but it could also be the presentation of a film, 
an interaction with a group, or an interaction with a teacher. 

Interactive instruction is instruction that engages the source 
and student in a dynamic process; the learner does something 
and the source does soemthing. The two are interrelated in that 
the things which the student does determine things which the 
source does. The use of the computer in interactive instruction 
represents a qualitative difference from the use of programmed 
instruction, and it is a significant innovation in the medical 
and dental school environment. Both programmed instruction 
and computer-aided instruction depart from the conventional 
use of media such as books, films, and television in that the 
student is continuously responding and receiving feedback, but 
it can also include in its branching these kinds of experiences. 
The flexibility of the CAI system, and its ability to branch, 
differentiate it significantly from programmed instruction. A 
CAI system processes according to a logic or strategy. It can do 
this selectively, with speed, and for a wide variety of responses 
from a number of students. It also can provide the learner with 
information about his performance at any point for either moti- 
vational or directional value. 

One of the simplest modes of interactive instruction is the 
problem-solving mode . In this mode the student is presented 
with a problem which he processes by defining the logic or 
procedure he will use to solve the problem. He then specifies 
this to the computer through a program which he prepares and 
provides with appropriate data. His program is a description of 
his solution. The computer processes the program and provides 
the student with the solution. If the solution is correct the stu- 
dent has described a satisfactory solution. A common purpose 
of the problem-solving mode is to build up the student’s skill 
level in the use of problem-solving procedures and appropriate 
concepts. 

In the problem-solving mode of instruction one of the 
teaching objectives is to determine the way in which the indivi- 
dual organizes . his solution. It is the logic of his approach to 
the problem that is critical. In order to utilize this mode of 
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instruction in its most common form, the student learns the 
language which the computer uses. However, it also is possible 
to develop programs that allow the student to solve problems in 
natural language. 

The inquiry mode of instruction is another mode which 
puts the student in control. He forms his questions using natural 
language, or in some more limited way. A thesaurus is often 
used to restrict the set of terms the student can use. The system 
responds by retrieving the information requested and presenting 
it to him at the terminal. In this mode the student has to learn 
very little about the way computers process, but he does have 
to learn how to ask the right kind of questions. Inquiries at a 
computer console must be formed in a way that allows the sys- 
tem to identify the critical elements in the data bank that are 
being sought. In the medical context, this kind of inquiry train- 
ing can be related to the mastery of a basic skill that is useful 
in diagnosis. 

The drill and practice mode is one of the most widely used 
modes at the present time. A recent summary* of CAI work 
reveals that the use of this mode of instruction in elementary 
school arithmetic and in college-level Russian can be very effec- 
tive. It is a mode that is readily adapted to a GAI system. 
However, it requires the commitment of personnel to the develop- 
ment of instructional programs. This mode is used primarily to 
teach students a particular set of skills that tie into a program. 
The student repeats a particular sequence of steps until he reaches 
a specified level of proficiency or uses up his allotted time. 

Another mode of interactive instruction is gaming. In this 
mode, a realistic problem is presented and the student is asked 
to make decisions: These decisions are processed and the outcome 
is communicated to the student. The objective of the game is to 
have the student learn a set of relationships between terms that 
define problems and those that describe the results when various 
decisions are made. In gaming, the computer is programmed so 
that the output associated with any input is realistic, in the 
sense that it could occur in the context to which the game per- 
tains. In actuality, the output generally is determined by a ran- 
dom generator which selects a result from a set of possiblities. 

Simulation is different from gaming in that the system has 



‘Suppes, P. & Morningstar, M. Computer-assisted instruction. Science, 1969, 166, 
343-350. 



27 



LEARNING IN THE DENTAL SCHOOL ENVIRONMENT 19 

stored within it a model of some real event system which the 
student is trying to understand. In a simulation, relationships 
are critical. These are represented by the model which is used 
to determine the consequences of decisions. The student knows 
the way in which events are presented and the ways in which 
the outcomes appear. What he does not know is the nature of 
the complex set of relationships between events, decisions, and 
outcomes. The system may or may not teach him the model as 
such, so that he knows it in formal mathematical terms. It is 
more likely to be used to build an intuitive dispositon which 
permits the student to perform "as if” he knew the formal 
relationships. Sometimes, however, it is the objective of simula- 
tion to actually teach the nature of the model which the system 
is using to provide the simulation experiences. 

The tutorial mode of instruction is a level of instruction 
that not only involves dialogue but also the other previously 
described modes on a selective basis. For example, the conse- 
quences of a student’s response to a question may be a drill 
and practice session to build up some critical skill. On the other 
hand, the student may be branched to a game or to a simula- 
tion. In the tutorial mode any or all of the other modes of 
instruction can be used, depending upon the needs of the learner 
and the objectives to be achieved. 



A COMPUTER SYSTEM FOR INSTRUCTION 

The Harvard Computer-Aided Instruction (CAI) System is 
shown in Figure 1. In addition to the computer, which in this 
case is an IBM S 360, Model 65, the system includes equipment 
to read the data from the cards, tapes, or disks that introduce 
instructional sequences into the system; the computer-aided in- 
struction language; and the students. The lower half of the figure 
describes the teleprocessing equipment used. It provides access to 
the system simultaneously from a number of different users in 
any location equipped with a telephone. A variety of student 
consoles can be used. Some provide only a typewriter as the 
input and output device. Others provide the student with audio 
and visual experiences through auxiliary devices that present 
slides and audio tapes under computer control. These presenta- 
tions are synchronized with the program so that particular 
slides and tape segments are utilized at appropriate points in the 
instruction. The system is capable of branching the student 
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l from one part, of the program to another, depending upon infor- 

t mation about the student and his past performance. 

I The following sections describe four CAI programs in the 

health sciences. Several are currently running on the Harvard 
| CAI System. The preparation of such programs serves a dual 

purpose, since the CAI system can be used to provide interactive 

■ instructional experiences and to do basic research on learning 

| and teaching. 

i □□□ 




Figure 1. The Harvard University computer-based interactive instruc- 
tional system. 






□ □ □ 
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Socratic Dialogue and Laboratory Simulation 
in Pathology* 

Dr, Earl E. Hellerstein, a pathologist at the Harvard Medi- 
cal School, has developed with the assistance of personnel at 
the Harvard CAI Laboratory a series of nine CAI pathology 
lessons. The course is presented on an IBM 1050 audiovisual 
terminal. In the course, slides are projected on a rear projection 
screen mounted on the surface of the desk at which the student 
works. The screen takes the place of a laboratory microscope. 
A tape recorder can be played with or without earphones for 
the convenience of the learner and the others present in the 
room. 

The primary objective of these lessons is to teach students 
to identify, from transparencies, critical characteristics of such 
organs as heart, artery, vein, lung, spleen, bone marrow, and 
lymph. In addition, the material was designed to teach students 
to connect morphological patterns, as represented by projected 
color transparencies of micorscopic slides of fungal diseases, with 
specific pathological lesions. The programs also teach various 
basic concepts in pathology such as predisposing factors and 
dissemination. In short, the student is taught a heuristic ap- 
proach to the processing of information presented in a trans- 
parency to help him determine what disease process is present 
in a tissue. 

More specifically, the nine course segments (Path 1 to Path 
9) were developed for the second-year medical student studying 
fungal diseases in the pathology laboratory. They were designed 
to produce six important types of behavior. The first is the 
identification , from transparencies, of critical information for the 
diagnosis of pathogenic entities. The second is to teach the 
recognition of variations in histological response to disease. The 
third is to describe tissue responses. The fourth is the specifica- 
tion of methods (tests) to confirm a diagnosis. The fifth is the 
naming of factors that determine the communication of disease. 
The sixth is the naming of predisposing factors involved in the 
acquisition of disease. 



*The following description is based on Technical Report No. 3, CAI— Socratic dia- 
logue and laboratory simulation in pathology, 1967, Office of Naval Research Con- 
tract No. N00014-67-A-0298-0003, Harvard University, by Earl E. Hellerstein, 
Terrill A. Mast, and Lawrence M. Stolurow. 
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The general teaching paradigm used is the Socratic dia- 
logue with supplemented branching strategies. A segment is 
defined by the existing course pattern. Each one begins with 
the student’s self-controlled viewing of slides of a series of mag- 
nifications of a section of a diseased organ. For example, in 
Path 1, the student secs a IX, 4X, 16X, 40X, 100X, and a 
200X phase magnification of the spleen infected by histoplasmo- 
sis. He is allowed to study each magnification as much or as 
little as he likes; he controls the time he uses. He has the 
option both here and at various other places in the course of 
viewing the entire set of transparencies at his own speed, at 
"flash” speed, or with no review. The system allows him to 
inquire or to retrieve information in which he is interested at 
any time. In addition, he interacts as an individual in a Socra- 
tic dialogue through his verbal responses and through the ques- 
tioning of the system which branches to materials that prompt 
and help him when he has difficulty. 

Having studied the transparencies, the student’s first task 
is to name the infected organ. If his identification of the organ 
is correct he moves ahead in the course, but if his identification 
is incorrect, he is automatically branched to a section of dialogue 
and a series of remedial frames requiring examination, analysis, 
and response. The objective of this series of frames is to con- 
centrate his attention on the critical characteristics or qualities 
of the section of tissue that reveal the class of organ, organ 
system, and specific tissue that he is viewing. This dialogue calls 
upon information which he should have learned earlier in his 
medical education but which he appears not to have applied to 
this task of identifying the organ from the transparency. The 
dialogue is designed to require retrieval and transfer of informa- 
tion previously taught, but not necessarily used in the context 
of a laboratory. In one way or another, depending upon the 
student’s response to each question about what he sees and what 
it means, he moves or is led to a correct organ identification 
before going on to a new one. 

A student-computer dialogue is reproduced below to illus- 
trate one of the paths which the interaction could take. As the 
course is currently programmed, different paths are possible, and 
the one that unfolds depends upon the student’s responses at 
several points. 

As this dialogue unfolds, the student has just viewed a 
sequence of transparencies— -in this case a series of IX, 4X, 
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16X, 40X, 100X, and 200X phase transparencies of spleen with 
histoplasmosis. 



Computer: DO YOU KNOW WHAT THIS ORGAN IS? 

Student: I think so 

C: THE INSTRUCTIONS TYPE YES OR NO MEAN: 

TYPE YES OR NO. 

S: yes 

C: (computer reprojects 40X magnification and types) 

40X AGAIN. OK, WHAT IS THE ORGAN? 

S: I think it is lymph node 

C: FAIRLY REASONABLE CHOICE, BUT WRONG. 

LET’S START FROM SCRATCH, (computer projects 
IX magnification) DO YOU THINK THIS IS A 
HOLLOW VISCOUS? YES OR NO? 

S: no 

C: THIS IS NEITHER MUCOSA NOR MUSCULARIS, 

HENCE THIS IS NOT GUT, UTERUS OR 
FALLOPIAN TUBE, URETER OR URINARY BLADDER. 
ALSO IT IS NOT HEART, ARTERY OR VEIN, 
(computer reprojects 200X magnification) THEN 
IT IS A SOLID VISCOUS. WAS IT LIVER? 

S: no 

C: CORRECT. THERE ARE NO PORTAL TRIADS, NO 

CORD CELLS ... SO IT IS. NOT LIVER. THE 

ORGAN CONTAINED MANY SINUSOIDS AS WELL 
AS A FAIR NUMBER OF RBC’S. THESE SUGGEST 
AN ORGAN OF THE RETICULOENDOTHELIAL 
SYSTEM; NAMELY THE . . . 

S: marrow 

C: WRONG. BONE SPICULES, FAT CELLS, AND 

HEMATOPOIESIS WERE ALL ABSENT. 

TRY AGAIN. 
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S: spleen 

C: THAT’S RIGHT. IT’S SPLEEN, (computer 

reprojects 16X magnification; this time it contains 

pointing arrows) UPPER ARROW - ONE OF THE 
FEW GERMINAL FOLLICLES IN THIS DIFFICULT 
H&E SLIDE. LOWER ARROW - ARTERIOLE. 

THESE HELP TO IDENTIFY THE ORGAN AS 
SPLEEN. NOTE THE VALUE OF LOW POWER. 

PRESS EOB WHEN READY TO MOVE ON. (EOB 
means "end of block” and signals "I am through" 
to the computer) 

Another possible line of dialogue follows. It would have 
been presented if the student had incorrectly answered "liver” 
when asked about the organ. 

Computer: (computer projects 40X magnification) 40X 
AGAIN. OK. WHAT IS THE ORGAN? 

Student: liver 

C: NAME ONE HISTOLOGIC FEATURE THAT 

CHARACTERIZES LIVER. 

S: portal triad 

C: CORRECT. HEPATIC CORD CELLS AND THE 

VARIOUS STRUCTURES IN THE PORTAL TRIAD 
ARE CHARACTERISTIC OF LIVER. THE LIVER ALSO 
HAS A CHARACTERISTIC NORMAL 
"ARCHITECTURE”. ARE THESE FEATURES 
VISIBLE IN THESE TRANSPARENCIES? 

S: no 

C: CORRECT. THE IDENTIFICATION OF LIVER THUS 

LACKS SOLID GROUNDING. NOT LIVER. DO YOU 
THINK THIS IS A HOLLOW VISCUS? YES OR NO? 

S: yes 

C: WRONG. THERE IS NEITHER MUCOSA NOR 

MUSCULARIS. HENCE THIS IS NOT GUT, 

UTERUS OR FALLOPIAN TUBE, URETER OR 
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URINARY BLADDER. ALSO, IT IS NOT HEART, 
ARTERY OR VEIN, (computer reprojects 200X 
magnification) THEN IT'S A SOLID VISCUS. WAS 
IT LYMPH NODE? 



S: 



no 



C: 



RIGHT. THE ORGAN CONTAINED MANY 



SINUSOIDS AS WELL AS A FAIR NUMBER OF 
RBC’S. THESE SUGGEST AN ORGAN OF THE 
RETICULOENDOTHELIAL SYSTEM; NAMELY, 
THE. . . 



S: 



spleen 



C: 



THAT'S RIGHT. IT’S SPLEEN. 



These two examples illustrate that the path a student takes 
depends upon what his responses are. However, his path de- 
pends upon not only the immediately preceding answer, as with 
a " right” or "wrong” message from the computer, but also 
upon the student’s previous behavior either in the course or in 
pretests (a student’s responses are stored by the computer in 
counters). The student is branched to one of the predetermined 
options that the author anticipated. In the examples above, the 
student who originally identified the organ as lymph rwde and 
the student who originally identified the organ as liver were 
both taken through approximately the same set of error diagnos- 
tic frames, but not precisely the same set in terms of the text. 
When student A answers "liver” the program places a "1” in 
his number 2 counter. At various points in the error-diagnostic 
set, the computer asks itself what this student’s original answer 
was. If his counter says he answered "liver” then it takes him 
one way. If the counter says he answered "lymph node” 
(stored as number 2, for example) then he goes to different 
material. If the counter says that he answered neither, he goes 
in still another direction. 

As this program is written, the student’s next task is to 
diagnose the disease of the organ. If the student thinks he 
knows the disease, the course may unfold in the following way. 



Computer: OK. WHAT DO YOU THINK THE DISEASE IS? 

(Depending upon his response the computer will 
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display different messages. For example, if his 
response does include the name of certain 
specified fungi, it will give him a list of certain 
broad categories of diagnosis. An incorrect category 
choice will cause the program to do one of two 
things, depending upon the answer he chooses. The 
first thing it can do is to continue the dialogue 
by going down the incorrect but logical path flowing 
from the student’s answer. The program 
participates for a time in the student’s stroll down 
a blind alley.) 

THE DISEASE FALLS IN WHICH CATEGORY? 

A. CIRCULATORY 

B. NEOPLASTIC 

C. BACTERIAL 

D. VIRAL 

E. PROTOZOAl. 

F. FUNGAL 

S: b 

C: IS THE NEOPLASM YOU HAVE IN MIND PRIMARY 

OR SECONDARY? 



S: primary 

C: THIS VIRTUALLY MEANS THAT YOU SAW EITHER 

A LUKEMIA OR A MALIGNANT LYMPHOMA. 
WHICH? 

S: lukemia 

C: NO PRIMITIVE HEMATOPOIETIC CELLS OR HUGE 

EXCESS OF ADULT WBC'S. NO MALIGNANT 
CELLS -NO NEOPLASM. SO -CHOOSE DISEASE 
CATEGORY FROM A TO F AGAIN. 



Alternatively, the second thing the program can do is to 
immediately tell the student he is wrong and give him the cor- 
rect answer. 
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Computer: THE DISEASE FALLS IN WHICH CATEGORY? 

A. CIRCULATORY 

B. NEOPLASMIC 

C. BACTERIAL 

D. VIRAL 

E. PROTOZOAL 

F. FUNGAL 

Student: e 

C: IF YOU HAD A PARTICULAR PROTOZOAL IN 

MIND, YOU COULD BE RIGHT. ACTUALLY, THIS 
IS A FUNGAL DISEASE. 



This program utilizes correction procedure. Whether the student 
chooses correctly on his first attempt or only after excluding 
others, he is told the correct answer or he .will determine that 
this is a fungal disease and will be shown the list that follows. 

Computer: WHICH ONE OF THE FOLLOWING FUNGAL 
DISEASES DO YOU PREFER? 

A. ACTINOMYCOSIS 

B. BLASTOMYCOSIS 

C. CANDIDIOSIS (MONILIOSIS) 

D. COCCIDIOIDOMYCOSIS 

E. CRYPTOCOCCOSIS (TORULOSIS) 

F. HISTOPLASMOSIS 



The student’s answer is treated the same way as his answer 
would have been had he answered affirmatively when asked if 
he knew the disease, and subsequently volunteered one of the 
above diagnoses. Suppose, for illustrative purposes, the student 
selects "blastomycosis” as the diagnosis. Then he would type 
"b”. The resulting dialogue is illustrated below. 

Student: b 

Computer: (computer projects 100X of a blastomycotic lesion 
and then types the following question ) WHAT 
DISEASE DOES THIS SLIDE SHOW? 
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S: BL astymocosis (In these programs, typing errors 

are ignored.) 

C: YES. THIS IS BLASTOMYCOSIS. BUT THE 

ORIGINAL WAS NOT. STUDY THIS TO CONTRAST 
WITH THE ORIGINAL. WHEN YOU ARE READY 
TO LOOK AT THE ORIGINAL AGAIN, PRESS EOB. 



This is, in our view, an interesting teaching strategy. It assists 
the student in his own error correction by letting him see the 
slide he mistook for the one he was asked to identify. 

In the example above, the student again makes an incor- 
rect diagnosis and receives a remediation treatment specific to 
his response. In this particular segment of the program, the 
student can err only three times. After the third round he is 
given the correct diagnosis and the program proceeds to new 
material. After the correct diagnosis of histoplasmosis is reached 
there follows a consideration of systemic histoplasmosis. 

Computer: IN A MINORITY OF CASES (USUALLY CHILDHOOD) 
THE DISEASE MAY NOT STAY LOCALIZED TO 
THE LUNG (AND ITS NODES). IT IS THEN SAID 
TO BECOME . . . 

Student: disseminated 

C: THAT IS RIGHT. IT BECOMES DISSEMINATED OR 

SYSTEMIC. ORGANS OTHER THAN THE SPLEEN 
SHOULD BE INVOLVED IN SUSTAINING 
HISTOPLASMOSIS. NAME ONE. 

S: spleen 

C: PLEASE NAME ORGANS OTHER THAN SPLEEN. 

S: lymph node 

C: OK. NAME ANOTHER. 

S: heart 

C: WRONG. TRY AGAIN. 

S: liver 
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C: 



OK. NAME ANOTHER. 



S: 



node 



C: 



YOU ALREADY NAMED THAT ORGAN. NAME 
ANOTHER. 



S: 



bone 



C: 



WE WILL GENEROUSLY ASSUME THAT YOU 



MEANT BONE MARROW. THE ORGANS OF THE 
RETICULOENDOTHELIAL (RE) SYSTEM INCLUDE: 
SPLEEN, LYMPH NODES, LIVER, AND BONE 
MARROW. TH ESE ORGANS ARE INVOLVED IN 
SYSTEMIC HISTOPLASMOSIS. PRESS EOB. 



This course segment, Path 1, goes on to point out that 
histoplasmosis may mimic tuberculosis, and that the histoplasmin 
skin test can be used as further evidence in differential diagno- 
sis. The course concludes with the reminder that other diseases 
may coexist with the infectious disease(s). 

It can be seen that the strategy used in this course takes 
advantage of the capability of the system to synthesize a variety 
of instructional sequences upon demand and dependent upon the 
student’s performance record. The demand is created by the 
retrieval of selected segments in the student’s response history 
along with appropriate demographic data. These data are then 
used as a basis for decisions about what happens next. The 
student may be visually or verbally prompted; he may be assisted 
in learning a critical discrimination or a set of diagnostically 
useful associations. 

One basic problem in interactive instruction is to determine 
how to synthesize from a set of possibilities the one teaching 
strategy that is most effective for a particular student. The pro- 
cess begins by speculating about factors that determine the gains 
students make in their performance on a post-test. This leads 
to the formation of hypotheses about critical factors in instruc- 
tion. Then a set of alternative conditions is described and two 
or more strategies are used with similar students. Data are col- 
lected to determine the effect that each strategy has on students. 
If it turns out that a particular strategy works well with particu- 
lar students, then the problem is to determine the means by 
which students are matched with the strategy that is best for 
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them. There also is the companion problem of deciding, on-line, 
what rules to change when the student does not perform accord- 
ing to expectation. 

Project TAPS: Teaching Anatomy 
with Programmed Schematics* 

Project TAPS is concerned with a research question relating 
to the improvement of learning and retention of anatomical in- 
formation through the type of visual display used in presenting 
information to the student. It is a study of the effectiveness of 
two different visual media in interactive instruction. More specif- 
ically, the visual materials are designed as perceptual aids to 
verbal learning. Schemata and representational diagrams are 
used to support a Socratic-style of dialogue which is designed to 
teach the topography, terminology, and concepts by means of a 
computer-aided instruction (CAI) system. The research aim is to 
determine the relative effectiveness of the visual information by 
comparing the performance of students after identical instructional 
programs which differ only in the visual materials used. The 
experimental comparison is made between a representational 
and a schematic drawing. Student, performance on a test is used 
to identify the condition which results in the more efficient re- 
tention of information. In addition, correlations will be computed 
to determine whether different kinds of students, in terms of 
aptitude profiles, learn better from one or the other of these 

visual aids. , 

TRIGEM. The CAI course developed for Project TAPS is 
called TRIGEM apd teaches the anatomy of the maxillary divi- 
sion of the trigeminal nerve. It consists of a total of 111 instruc- 
tional frames and requires about one and one-quarter hours of 
student time. It is currently operational on the Harvard CAI 
system. The course can be accessed by either a standard teletype 
(model KSR 33) or an IBM 1050 terminal connected to a 
103A2 data set. TRIGEM presents the student with information, 
both visual and verbal, about the trigeminal nerve. One objective 
of the program is to teach the student about nerves in general. 
Others are to teach the branch configurations of the trigeminal 



’This project was made possible by U.S. Public Health Service Contract No. PH 
108-69-18. 
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| nerve, the names of the various divisions of the nerve, and the 

| number of divisions that occur at each level. 

Two programs, PD and PF, have been prepared and they 
differ only with respect to the supporting visual material. Pro- 

□ □□ 






Figure 2. The two representational diagrams used in Program PF of 
the CAI course TRIGEM. 



□ □□ 
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gram PF uses two representational diagrams (Figure 2) taken 
from Gray’s. Anatomy. Program PD uses a schematic diagram 
and identification key (Figures 3 and 4). These materials were 
dry-mounted on heavy cardboard and made available to students 
at the console. 

Objectives. The main objectives of the program are as 
follows: (1) to enable the student to diagram the structure of 
the maxillary division of the trigeminal nerve; (2) to enable 
the student to label his diagram with (a) all the appropriate 
anatomical terms for the nerves, (b) the regions from which 
each branch originates, (c) the region through which it passes, 
and (d) the region or organ it supplies; and (3) to enable 
the student to answer verbal questions about each area described 

above. t 

Both programs are divided into four sections. The first 

section, which provides a description of the nerves in gen- 
eral, makes it possible for the student to describe the nerve 
tissue. It introduces the student to the trigeminal nerve and 
emphasizes the maxillary division. The objective of the second 
section is to teach the student to name the branch configurations 
of the maxillary division and to sketch the configurations, so 
that when he makes his sketch he will also be able to identify 
each of the several component structures of the nerve. The 
third section teaches the student the names of the nerves and 
the areas they supply. Upon completion of this section the 
student should be able to name each of the nerves and the 
areas they supply. Upon completion of the fourth section of 
the program, the student should be able to name the regions 
through which each of the branches of the maxillary division 
passes. 

Types of Frames. Two basic types of instructional frames 
are used. One is denoted by the abbreviation RD ( Read ) and 
the other by QU ("Question”). These terms refer to the codes 
used in C AHA N, the CAI language used on the Harvard CAI 
system. They give the system instructions for processing the 
text that follows each. An RD frame requires the student to 
simply read a paragraph. When he is ready to continue he 
presses the EOB— end of block— key which signals the sys- 
tem to proceed with the course. Some RD frames require the 
student to sketch some region of the trigeminal nerve. The 
booklet provided for this purpose contains black pages for the 
drawings, alternating with pages on which the correct configura- 
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□ □ □ 




Figure 3. The schematic diagram used in Program PD of the CAI 
course TRIGEM. 
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tions are printed.* The student draws on a blank page and 
then turns it over to disclose the correct answer. When he is 
satisified he knows a configuration, he presses the EOB key and 
the computer proceeds to the next frame. The student may take 
as much time on a question as he wishes before moving on. He 
is also given immediate knowledge of results. As soon as he 
responds he is given a performance standard against which to 
check his work. 



DIAGRAM KEY 



Symbol 


What the symbol represents 


i ~~i 




Maxillary nerve 


Nerve name 


Rounded forms 




(OA) 


Ganglion 


O O o ° 0 0 ° ° 


Middle meningeal nerve 




Pterygopalatine nerve 


mwwwvm^Mi 


Facial branches 





Superior alveolar branches 




Zygomatic nerve 



BREAKS IN THE CENTER LINES CONTAIN THE NAMES 
OF THE REGIONS THROUGH WHICH THE MAXILLARY PASSES. 



Figure 4. The identification key used with the schematic diagram 
shown in Figure 3. 

□ □□ 



♦Hard copy answer booklets were used to minimize student console costs, although 
it would have been more convenient and efficient to use a console equipped with a 
slide carousel or CRT. 
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Sample RD frames from section one are shown below: 

Computer: NERVES ARE LONG WHITISH STRINGS OF CEl L 
BODIES AND THEIR "APPENDAGES” WHICH 
CONDUCT MESSAGES FROM ONE PART OF THE 
ANATOMY TO ANOTHER. THE NERVE 
APPENDAGES ARE THE LONG PARTS WE 
USUALLY REFER TO WHEN WE SPEAK OF NERVES. 

****** **** 



Computer: NOW, WITHOUT LOOKING AT THE DIAGRAM, 
SKETCH THE DIAGRAMMATIC CONFIGURATION 
FOR ALL THE PTERYGOPALATINE NERVES. DRAW 
ON PAGE 11, AND THEN COMPARE YOUR 
DRAWING TO THE ONE SHOWN ON PAGE 12 OF 
THE BOOKLET. 



Whenever the student is ready to move on, he presses the EOB 
key. 

A QU frame requires the student to type a response. This 
response may be to fill in a blank or answer a direct question. 

j S correct ’ the computer responds by typing 
OUKKtLl and then proceeds to the next frame. If the answer 
is correct but misspelled, the computer responds by calling the 
student’s attention to his misspelling indirectly, by typing the 
word correctly in the response telling him he has answered 
the item correctly. This is called error-contingent prompting 
strategy and is used to preserve the student’s morale and moti- 
vation- When the student’s answer is incorrect, the computer 
tells him he is, incorrect, gives him the correct answer, and 
proceeds to the next frame. This is called non-correct strategy 
It was used throughout the program for all students, to main- 
tain consistency for research purposes. Alternatively, the com- 
puter could have been programmed to require the student to 
make the correct response before allowing him to proceed to 
the next frame, as it was in the pathology program. 

An example of a QU frame is as follows: 



Computer: LOOK AT THE DIAGRAM. THE 

PTERYGOPALATINE NERVES APPEAR TO FORM 
THE . . . GANGLION. ACTUALLY THEY PASS 
RIGHT THROUGH THE GANGLION AND DO 
NOT FORM IT. 
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Student: pterygopalatine 



C: 



CORRECT 



This example illustrates the way in which the visual aid serves 
as a prompt for the correct response. The question calls the 
student’s attention to the critical information, and the sequence 

used can aid in forming discriminations. . 

Another example of a QU frame, to which the student giv s 
an incorrect answer, is shown below. 

Computer: ONE BRANCH OF THE P°S TE RIOR SUP E R10R 

NASAL BRANCHES IS LONGER THAN THE REST. 
THIS BRANCH IS NAMED ON THE DIAGRAM AND 
IS THE . . . 



Student: pharyngeal 



C: 



INCORRECT. THE CORRECT ANSWER IS THE 
NASOPALATINE. 



Note that the strategy used here is quite different from the one 
used in the pathology program, although the same one could 
have been applied with this type of visual material. 

Tests Before taking TRIGEM, students are given a test 
question booklet and an answer booklet. The test includes verbal 
items covering knowledge of the structure, function, and termi- 
nology of the trigeminal nerve. There is no difference in the 
pre- and post-tests. This procedure makes it possible to select 
frames from the total program that the student does not know 
although this was not done in the study being described. 

All students also fill out an attitude questionnaire before 
and after the written test. This questionnaire measures the stu- 
dent’s attitude toward CAI. It. contains 15 statements and re- 
quires about one minute to complete. 

Results. To date six students at Tufts University Dental 
School have taken the course, three Program PD and three 
Program PF.* Data on their performance have been used to 
improve the answer-processing procedures. When differences 
between pre- and pst-test scores for both groups were compared 

. With the cooperation of Dr. William M. Feagans at Tufts University Dental School 
five teletypes connected by telephone lines to the Harvard system were installed m 
a trailer adjacent to the school. 
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it was found that the group which took Program PD (schematic 
diagram and key) showed the larger gains on parts 1 and 2 
and in total score. The group that studied with the representa- 
tional drawing achieved larger gains on part 3, the section that 
teaches the names of the nerves and the areas they supply. 
Any further interpretation must await the completion of a second 
study, now in progress, utilizing a large student sample. 



&■ 



S/f 

t: 




CAI in Ophthalmology* 

The Vision Information Center (VIC), a system that com- 
bines instruction and information retrieval, was established in 
Boston on June 15, 1966 by Dr. Robert D. Reinecke and his 
associates at the Massachusetts Eye and Ear Infirmary and the 
Harvard Medical School, with the assistance of the Harvard CAI 
Laboratory. The philosophy behind the system is to give the 
user direct access to bibliographic information, thereby eliminat- 
ing the intermediate step of consulting an information specialist. 
This is made possible by making available at the computer 
terminal instructional programs in ophthalmology which aid the 
user in specifying his request for information. 

The user seats himself at a console, either an IBM 1050 
terminal or a teletype. He is first given the option of taking a 
program called "Terminal” but if he is already familiar with 
the operation of the console he can bypass this program and go 
directly into the search mode. In addition to information re- 
trieval, the user may also elect an instructional program in 
various areas of ophthalmology, e.g., strabismus, glaucoma, at 
any point in his search. While these CAI programs are self- 
contained units and may be used alone for other purposes, they 
are also intended to serve a user of the VIC system. 

The CAI programs are ^available whenever a user needs 
help in making his request for information more precise. If, for 
example, he were to ask for information on the eye, he could 
retrieve the whole data base. The system could, however, display 
a diagram of the eye and ask the user to identify the parts in 
which he is interested. After recording that response, the system 
could ask him whether he is interested in function or structure. 



*This project is supported by U.S. Public Health Service Contract PH 43-66-911. 
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If the answer is function, it could then determine whether he is 
interested in normal or pathological functioning. Through such 
a procedure the information needs of the user are determined at 
a level which permits the system to identify an appropriate in- 
structional program. 

The next step would be to teach definitions and concepts 
relating to the user’s area of concern. The idea is that some 
instruction preceding an inquiry from the data base can provide 
a more appropriate list of titles of publications than untutored 
inquiry. Also, instruction may be sought after articles have been 
identified so that the information they contain can be learned 
more efficiently. 

When the user has narrowed down his initial inquiry he 
enters a thesaurus number from a manual available at the con- 
sole. The system then types back the number of references 
available under that term. It then asks whether he v T ishes to 
continue or wants to select another term as well,. 

If he wants to continue, he enters a second number. This 
procedure can be used to narrow the search to the overlapping 
set. At any point the user may ask for the entire list of refer- 
ences. If the list contains less than twenty references, the entire 
list is printed »ut at the console. If there are more than twenty, 
the first ten are typed out at the console and the rest are mailed 
to him. 

The VIC system has been described by Drs. Mary M. 
Eickhorn and Robert D. Reinecke in the Journal of Chemical 
Docmentation (vol. 9, No. 2, May 1969). This article includes 
a flow chart (Figure 5) showing the relationship between CAI 
and IR (information retrieval) as it is being developed in this 
system. 

The largest CAI programs relating to the VIC system are 
on basic ophthalmology and glaucoma. At the present time the 
student must request these courses. Thesaurus code numbers 
have been assigned to frame sequences dealing with specific 
subjects. This code is the link between the data bank and pro- 
gram segments. For example, if the user wants to know about 
the technique of ophthalmoscopy, the frames of the basic ophthal- 
mology course which explain ophthalmoscopy will then be pre- 
sented to him if he signs on for CAI and enters that term. In 
this way the user can readily receive both instruction and a bib- 
liography on the subject he has selected from the thesaurus. 

Several users from remote locations have used the VIC sys- 
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tem and residents in ophthalmology have taken the CAI courses. 
The system appears to be effective in principle and it is being 
developed further to improve the techniques and to expand the 
data base. 



£ 

t 

}' 





Figure 5. Design of the VIC computer- user interaction; following the 
system explanation, the user has the option of the selecting computer- 
assisted instruction (CAI), literature retrieval (IR), or a combination of 
both (CAI-IR). 



□ □□ 



CAI in Hematology 

In the case of CLOT, an instructional program in hema- 
tology, CAI is used to simulate a clinical experience. This pro- 
gram was developed by Dr. Daniel Deykin and Miss Barbara 
Gilcrest, both of the Harvard Medical School. The following 
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frames illustrate the strategy they are usipg to develop this 
course. 

FRAME 1 



Student: s53/clot 

Computer: YOUR NAME IS STUDENT 53. 

HELLO. I AM THE JUNIOR INSTRUCTOR ON THE 
HEMATOLOGY STAFF. TO WHOM AM I SPEAKING 
PLEASE? TYPE IN YOUR LAST NAME, THEN 
PRESS EOB. 

S: smith 

C: AND YOUR FIRST NAME. 

S: john 

C* THANK YOU. I THINK, DOCTOR SMITH, WE 

SHOULD PLAN TO SPEND ABOUT 45 MINUTES 
TOGETHER. HAVE YOU READ THE PAGE OF 
PROGRAM INSTRUCTIONS AND YOUR NOTES FOR 
LECTURES XIII AND XIV? 

S: yes 

C: GOOD. THEN LET’S BEGIN! 

An attempt at personalization has been made in the sign-on 
procedure. The student’s name is stored in a buffer and used 
at different points in the program. 

FRAME 2 

Computer: AT 11 P.M. ON CHRISTMAS DAY, A 23 YEAR OLD 
AGITATED, WEEPING NURSE IS BROUGHT TO THE 
EMERGENCY WARD WHERE YOU ARE ON DUTY. 
SHE IS BLEEDING FROM HER NOSE AND GUMS, 
AND HAS MULTIPLE ECCHYMOSES. HOW WOULD 
YOU PROCEED? USE THE KEYBOARD TO ENTER 
THE STEPS YOU FEEL SHOULD BE TAKEN FIRST. 
PLEASE USE ONLY WORDS FROM YOUR 
VOCABULARY LIST. 
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This frame presents a verbal description of a medical episode 
that defines a problem for the student. To make the task man- 
ageable at this point, the student is given a list of terms to 
use in making his responses. In this way his vocabulary is 
delineated and the processing simplified. 

Frames 3, 4, and 5 show how the student is prompted to 
respond with information he has been taught but has not used 
in a clinical setting. The prompting techniques are being de- 
veloped. Some also involve the use of discrimination techniques 
so that inaccurate or inappropriate responses are immediately 
identified by the system for the student, but this strategy has 
not been tested as yet. 

FRAME 3 

Computer: PRESS EOB AFTER EACH CHOICE AND WAIT FOR 
THE GREEN PROCEED LIGHT BEFORE ENTERING 
ANOTHER. PRESS EOB ALONE TO END YOUR 
LIST. 

Student: detailed history 

C: CORRECT. ANYTHING ELSE? IF NOT, PRESS EOB. 

S: physical exam 

C: CORRECT. A DETAILED HISTORY AND 

PHYSICAL EXAM ARE ALWAYS FIRST IN 
EVALUATING A PATIENT. SPECIFICALLY, WHAT 
TYPE OF INFORMATION WOULD YOU WANT TO 
OBTAIN IN TAKING THIS GIRL’S HISTORY? 
CONSTRUCT PHRASES FROM THE VOCABULARY 
LIST. FOR EXAMPLE, “MOTHER’S MAIDEN NAME,’’ 
IF THOSE WORpS APPEARED ON THE LIST. 

PRESS EOB AFTER EACH AND WAIT FOR MY 
COMMENT BEFORE ENTERING THE NEXT PHRASE. 
PRESS EOB ALONE TO TERMINATE THE LIST, 

AS BEFORE. 

FRAME 4 

Student: previous response to trauma 

Computer: GOOD. AND? 
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S: 


* 

joint bleeding 


C: 


YES, THIS INFORMATION COULD BE 
OBTAINED IN THE HISTORY. WHAT ELSE? 


S: 


drug history 


C: 


CORRECT, IS THAT ALL? 


S: 


family bleeding problems 


C: 


NO, AT LEAST 1 NEVER PHRASED IT THAT WAY. 
TRY AGAIN. 


S: 


hereditary bleeding disorders 


C: 


VERY GOOD. CONTINUE. 


FRAME 5 




Student: 


dietary deficiency 


Computer: 


VERY GOOD. CONTINUE. 


S: 


previous illness 


C: 


NO, SORRY. IF YOU ARE PRETTY SURE OF YOUR 
ANSWER, YOU SHOULD PARAPHRASE IT, 
HOWEVER. OTHERWISE ENTER SOMETHING ELSE. 


S: 


location of skin bleeding 


C: 


YES. WHAT ELSE? 


S: 


location of joint bleeding ' 



O 

ERIC 



Review and clarification are introduced in Frame 6. There 
is a good deal of research to suggest that guided review is an 
effective procedure. The problem in the CAI setting is one of 
using it as effectively as possible. This is where the requirement 
for research to establish a rule .becomes evident. We need to 
know the factors that go into the. "if . . . then . . .7 state- 
ment for the use of. review. Similarly, we need to study the 
use of preview. 
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FRAME 6 

Computer: YES. WHAT ELSE? 



Student: 


1 don’t know 


C: 


WE SEEM TO HAVE SOME COMMUNICATION 
PROBLEMS, AND 1 DON’T WANT TO INSIST ON 
SPECIFIC WORDING. FOR EFFICIENCY, ALLOW ME 
TO SUMMARIZE THE DESIRED INFORMATION. 

1. ONSET OF SYMPTOMS 

2. RESPONSE TO TRAUMA OR SURGERY 

3. CHARACTERISTICS OF BLEEDING (PRIMARY 
OR SECONDARY) 

4. OTHER ILLNESS (e.g.: RENAL OR LIVER 
DISEASE, CANCER) 

5. FAMILY HISTORY OF BLEEDING 

6. DIETARY ABNORMALITIES 

7. HISTORY OF DRUG INGESTION AND 
MEDICATIONS 



Isolation is accomplished in this program in Frame 7 by 
asking the student what he would especially look for during the 
physical examination. Other techniques are used in CAI pro- 
grams, e.g., underlining, capitalization, typing in red. 



FRAME 7 




Computer: 


WHAT WOULD YOU ESPECIALLY LOOK FOR 
DURING THE PHYSICAL EXAM? PROCEED AS 
BEFORE. 


Student: 


location of all bleeding areas 


C: 


YES, THIS INFORMATION IS IMPORTANT. 


S: 


malnutrition 


C: 


GOOD. NO ONE MENTIONS THIS. 


S: 


malignancy 


C: 


GOOD. NO ONE MENTIONS THIS. 
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S: bruise 

C: GOOD. NO ONE MENTIONS THIS. 

Note, in Frame 8, that reliance is occasionally placed upon 
the request for paraphrasing of responses. This is a middle 
position between saying right or wrong. It is employed while 
we accumulate data from students that can then be used to 
make the system better able to act as if it understood the stu- 
dent. This ability to constantly and immediately update the 
program is not to be underestimated. Also involved is student 
authorship. The responses the system collects provide data for 
use by the author in modifying the program. Student feedback 
to the author is necessary; it is one way to upgrade the system. 



FRAME 8 

Student: jaundice 

Computer: YES, AS A SIGN OF OTHER ILLNESS. AND? 

S: deformity 

C: NO, AT LEAST I NEVER PHRASED IT THAT WAY. 

TRY AGAIN. 

S: petechiae 

C: GOOD. NOONE MENTIONS THIS. 

S: pulse 

C: YES, THIS COULD INDICATE A MAJOR BLOOD 

LOSS. 

S: blood pressure 

C: YES, THIS COUlD INDICATE A MAJOR BLOOD 

LOSS. 

S: evidence of previous surgery 

NO, SORRY. IF YOU ARE PRETTY SURE OF YOUR 

53 
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ANSWER, YOU SHOULD PARAPHRASE IT, 
HOWEVER. OTHERWISE ENTER SOMETHING ELSE. 

Another use of review is illustrated in Frame 9. Frames 
10-14 illustrate the program and Frame 15 shows the summary 
given to each student. 

FRAME 9 

Computer: YOU WILL PROBABLY DO WELL IN THE CLINIC. 

HOWEVER, LET ME JUST SUMMARIZE WHAT ONE 
SHOULD LOOK FOR DURING THE PHYSICAL 
EXAM. 

1. LOCATION OF BLEEDING (SKIN, JOINTS) 

2. JOINT DEFORMITY 

3. OTHER ILLNESS (e.g., UREMIA, CANCER) 

4. EVIDENCE OF MALNUTRITION 

UNFORTUNATELY THE NURSE WAS TOO 
AGITATED TO GIVE A MEANINGFUL HISTORY, BUT 
HER ROOMMATE SAID THE PATIENT HAD NEVER 
BLED BEFORE AND SHE KNEW OF NO FAMILY 
HISTORY OF BLEEDING. SHE ALSO SAID THE 
PATIENT HAD BEEN UNUSUALLY DEPRESSED 
DURING THE PREVIOUS WEEK. WHAT 
SCREENING LABORATORY TESTS SHOULD BE 
DONE? FOR CONVENIENCE, ENTER THE 
CORRESPONDING LETTERS (A-M), LEAVING A 
SPACE BETWEEN ENTRIES. 

FRAME 10 

Computer: WHICH STAGE OF THE CLOTTING SEQUENCE 
DOES THE THROMBIN TIME TEST? (FIRST, 

SECOND, THIRD) 

Student: third 

C: CORRECT. IN THIS CASE THE THROMBIN TIME IS 

NORMAL, IMPLYING FIBRINOGEN CONVERSION 
IS INTACT. THE GIRL HAS NO PAST HISTORY OF 
BLEEDING, NOR IS THERE A FAMILY HISTORY OF 
BLEEDING. UNDER SUCH CIRCUMSTANCES, 

HER SEVERE BLEEDING SUGGESTS AN ACQUIRED 
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S: 

FRAME 11 

Computer: 

Student: 

C: 

S: 

C: 

FRAME 12 
Computer: 



DEFECT. WHAT KINDS OF ACQUIRED BLEEDING 
DEFECTS ARE CONSISTENT WITH THIS CLINICAL 
PICTURE? ENTER YOUR CHOICES ONE AT A 
TIME. PRESS EOB AFTER EACH AND WAIT FOR 
THE PROCEED LIGHT BEFORE ENTERING THE 
NEXT. PRESS THE EOB KEY ALONE TO 
TERMINATE YOUR LIST. 



drug 



CAN YOU GIVE ME AN EXAMPLE? 
aminocaproic acid 

“AMINOCAPROIC ACID” IS NOT ON MY LIST OF 
CORRECT ANSWERS. YOU MAY HAVE MISSPELLED 
SOMETHING, USED A WORD NOT IN MY 
VOCABULARY, OR MADE A MISTAKE. TRY AGAIN. 



coumarin 

YES, COUMARIN OR WARFARIN (THE GENERIC 
NAME) CAN CAUSE A BLEEDING DEFECT. 



I FEEL I MUST URGE YOU TO BE MORE 
COMPREHENSIVE IN YOUR APPROACH TO SUCH 
QUESTIONS. FOR NOW: LET ME GIVE YOU A 
CHECK LIST OF ACQUIRED BLEEDING SYNDROMES 
WHICH MUST BE CONSIDERED. NOT ALL 
PERTAIN TO YOUR PATIENT, OF COURSE. 

1. ACQUIRED INHIBITOR OF BLOOD CLOTTING 

2. HEPATIC DYSFUNCTION 

3. INGESTION OF AN ANTICOAGULANT DRUG 

4. DISSEMINATED JNTRAVASCULAR 
COAGULATION (i.e., CONSUMPTION OF 
COAGULATION FACTORS) 

5. FIBRINOLYSIS 

6. DILUTION OF COAGULATION FACTORS (AS 
WITH MORE THAN 8 UNITS OF STORED 
BANK BLOOD) 



LEARNING IN THE DENTAL SCHOOL ENVIRONMENT 47 

7. RENAL DISEASE (UREMIA) 

8. IDIOSYNCRATIC DRUG REACTION (APLASTIC 
ANEMIA, THROMBOCYTOPENIA) 

I WOULD LIKE TO STRESS THAT SUCH ACQUIRED 
DEFECTS ARE USUALLY MULTIPLE AND HAVE AN 
ACUTE ONSET, BUT MAY BECOME CHRONIC. 
DISSEMINATED INTRAVASCULAR COAGULATION 
(4) WOULD BE UNLIKELY IN THE FACE OF A 
NORMAL PLATELET COUNT, HOWEVER. 



FRAME 13 

Computer: NEXT WE MUST DISTINGUISH AMONG THE 

ABOVE POSSIBILITES. WHICH DISORDER WOULD 
BE EITHER DIAGNOSED OR ELIMINATED BY 
MIXING A SMALL AMOUNT OF THE PATIENT’S 
BLOOD WITH A LARGER VOLUME OF NORMAL 
BLOOD IN COAGULATION TESTS? 

Student: acquired inhibitor of blood clotting 

C: VERY GOOD. 

FRAME 14 

Computer: WHAT WOULD YOU LIKE TO SEE HAPPEN TO 
THIS PART OF THE HEMATOLOGY COURSE IN 
THE FUTURE? 

1. CONTINUE COMPUTER TEACHING, 
INCREASING THE NUMBER OF PROBLEMS ON 
CLOTTING DISORDERS AND/OR INCLUDING 

’ PROBLEMS ON OTHER COURSE MATERIAL. 

2. CONTINUE COMPUTER TEACHING, KEEPING 
THE NUMBER OF PROBLEMS CONSTANT. 

3. DISCONTINUE COMPUTER TEACHING. 

4. NOT SURE. 

Student: 1 



DID THE PROGRAM FORMAT ITSELF HINDER 
YOUR LEARNING? 

1. NO 

2. YES 

3. NOT SURE 
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S: 3 

C: THANK YOU, JOHN SMITH. 

FRAME 15 

Computer: WOULD YOU LIKE TO SEE HOW YOUR 

PERFORMANCE HAS BEEN RECORDED FOR THIS 
SESSION? IT WAS DONE FOR PROGRAM 
EVALUATION, NOT FOR GRADING PURPOSES. 

Student: yes 

C: ALL RIGHT. 

CORRECT 7 

WRONG 14 

I DON’T KNOW 1 

TIME SPENT WAS 60 MINUTES. 

PLEASE TAKE THE TYPEWRITER OUTPUT 
WITH YOU SO THERE WILL BE NO SAMPLE FOR 
STUDENTS IN THE FOLLOWING SESSIONS. NOW 
TYPE “SIGN OFF.” 

S: sign off 

C: LINE IS SIGNED OFF 

This program was used for the first time this semester. 
Data on student performance, when analyzed, will be used 
mainly to improve the program. This type of program provides 
the student with the opportunity to make the transition from 
the basic sciences to the clinical use of information. The transfer 
of learning problems that are involved in this conversion of 
information is a critical phase of dental and medical school 
experience which has received too little attention and offers 
much promise. 

TRANSFER RESEARCH 

Psychologically, the area of transfer of learning is very 
important. There are many findings from transfer studies that 
provide useful do’s and don’t’s for the programmer. We plan 
to examine these and to put many of them to the test within 
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instructional programs in the health sciences. 

For example, one phenomenon of interest is proactive in- 
terference. This is a phenomenon that relates to the effects of 
early learning upon the subsequent learning and retention of 
similar material. The results of research indicate that earlier- 
learned material that is similar to newly-learned material in- 
terferes with the retention of the newly-learned information. It 
also has been shown that overlearning can reduce interference. 
The question is whether it is possible to develop some rules for 
use with CAI to raise the student’s level of mastery to the 
point where this kind of interference is minimized. The rationale 
for this approach comes primarily from many separate studies 
conducted in the laboratory. Consequently our research would 
provide an integration of knowledge in the field of learning as 
well as relate to a practical problem of instruction. Such con- 
cepts as proactive interference, learning how to learn, and over- 
learning all need to be studied and related to the problem of 
developing instructional strategies. 

SUMMARY 

The needs of dentistry and the health sciences in general 
suggest that educational technology could be of great practical 
value. The work accomplished thus far at different CAI labora- 
tories on problems of instruction is encouraging. While the ex- 
perience to date is limited, the concepts and approaches seem 
to fit current needs and to provide an effective solution which 
the students probably like more than the faculty. The accom- 
plishments to date have been modest and were not easily 
achieved. Nevertheless, the results are sufficiently positive in 
terms of student performance to suggest that there is consider- 
able promise in this approach, not only for its contribution to 
interactive instruction but also for its contribution as a catalyst 
to all forms of instruction in a dental and medical school 
environment. 

It is apparent that the form and operation of educational 
programs in the health sciences are going to be considerably 
changed over the next few years; instruction will become more 
individualized and interactive. It would seem that computer- 
aided instruction could make a substantial, positive contribution 
to this kind of change. Hopefully, present economic constraints 
will not impact too severely on the further development of what 
appears to be a promising set of new solutions and. developments. 
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COMING TO TERMS WITH CAl 

In order to put into perspective the field of computer-assisted 
instruction (CAl) and its environs, consider the following defini- 
tions.* Technology is the systematic application of organized 
scientific knowledge to practical tasks. Technology, in these 
terms, is in no way equated with hardware. Rather, technology 
in this sense is synonymous with the well-known term applied 
science. 5 , 

Of all the practical tasks that could concern us, the scope 
of this discussion is limited to education. Therefore, Educational 
Technology is the application of technology to education. 

While many areas of education interest me, the area of 
instruction draws my greatest attention. The following definition 
recognizes that there are only four possible components in any 
instructional setting: students, subject matter, materials and 
teachers— the student being the only one necessary for learning. 
Instructional Technology is the use of technology to improve the 
student’s interaction with his subject matter, materials and 
teachers. This definition assumes that unless the interaction is 
improved it is not worthy of the title. 

In order to define CAl, the only word substitutions neces- 
sary are ''CAl” and "computer” to form the definition: CAl is 
the use of a computer to improve the student’s interaction with 



‘Formulated, in private discussion, by P.M. Beatts, IBM Education Research, San 
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his subject matter, materials and teachers. This definition should 
lay to rest the question whether good teachers will be displaced 
by CAL 

It is appropriate to ask, "In what ways can a student 
interact with his subject matter, materials and teachers?” The 
following list of eleven "modes of interaction” is offered as a 
working basis for this discussion and can be, I think, a helpful 
guide for planning and evaluation. 

1. drill 

2. practice 

3. problem review 

4. diagnosis and prescription 

5. tutorial 

6. gaming 

7. simulation 

8. factfinding 

9. computation 

10. logical problem solving 

11. exploration 

This list also provides an operational test for defining the 
term "computer-managed instruction” (CMI). If diagnosis and 
prescription are involved in the course, is the instruction itself 
contained in the computer program? If the answer is yes, then 
the course is CAI; if no, then it is CMI. 

While space will not permit discussion of each of the above, 

I would like to define several terms in order to show the differ- 
ent requirements imposed on instructional systems. Drill, for 
example, is the formation of correct habits under routinized 
conditions while practice implies an accustomizing or the acquir- 
ing of proficiency under a wide variety of conditions. At the 
other end of the list, exploration is characterized on the part of 
the student by an inquiring or "what if?” attitude. This leads 
to the conclusion that, under conditions of exploration, one must 
grant to the learner a great deal of freedom to probe freely in a 
subject matter. 

The requirements for CAI at this level, including course 
organization, terminals and operating systems, resemble those of 
information retrieval systems rather than those previously asso- 
ciated with teaching machines. It should also be pointed out that 
these eleven interaction types are not mutually exclusive; that is, 
one could combine a number of these types to form dynamic 
courses. What was previously considered elementary in CAI, 

BO 



52 GAI: TECHNICAL ASPECTS AND RESOURCES 

i.e., practice, now takes on a very different appearance under 
these conditions. Consider the following examples of experimental 
work being conducted at the IBM Education Research Departmcr 
at San Jose, California. 

PUTTING THE TERMS TO WORK 

Learner-controlled Statistics 
Ralph Grubb and Tom Weik 

This course segment of two chapters (Measures of Central 
Tendency and Measures of Dispersion) represents an attempt to 
combine certain qualities of exploration with those of tutoring, 
computation, and simulation, into a series of maps. Thus, the 
student can chart his own path through introductory statistics. 
The reasons for putting the learner rather than the program in 
control are the scarcity of authors who know how to write 
courses that recognize individual differences adequately; the wide 
differences in students’ background and motivation; and the need 
for making students responsible for their own actions. 

The current investigation is being carried out on an IBM 
1500 instructional system which was designed for research and 
development in CAL 

The system consists of an IBM 1800 central processor with 
disk cartridges for active course files and the system monitor, 
tape units for bulk storage of curriculum and student records, a 
station controller and video buffer, and the following input-output 
(I/O) units: typewriter, cathode ray tube (CRT), keyboard, light 
pen, image projector and an audio response unit. 

The course segments reported here use only the CRT, key- 
board, and light pen as I/O devices. 

The system monitor utilizes an author entry language 
known as Coursewriter II. This language permits subject matter 
experts who are naive concerning computers to construct their 
courses under a variety of teaching techniques. Later in this dis- 
cussion I will have more to say regarding developments in the 
area of author languages. 

Method \ When a student registers for this course via the 
CRT terminal, the first display he encounters is a map of the 
statistics course with appropriate directions for its use. 

The map consists of a series of boxes, connected in various 
ways, and inscribed with topics or concepts that the student can 

L- 



CAI: TECHNICAL ASPECTS AND RESOURCES 53 

access in order to route his way through the course. To do 
this, the student points his light pen at the area of the map 
that interests him. Immediately, his screen is erased and he is 
confronted with a more detailed map of that subsection of the 
course. Similarly, as he makes successive choices with the light 
pen he proceeds deeper into successive levels of the course. At 
the lower levels of the structure he may enter on the keyboard 
constructed responses to questions or problems posed through 
the computer. If he chooses, he can also point his light pen at 
any time to one of several target areas to the side of the screen 
that will return him to the previous map, main map, or a 
glossary. 

In addition, there are usually other target functions repre- 
sented on the screen which will transfer the student laterally to 
a number of related topics in the course. This would suggest 
that the "map” approach is more analogous to a network than 

□ □ □ 




Figure 1. IBM 1500 instructional system. 

□ □□ 

62 



54 CAI: TECHNICAL ASPECTS AND RESOURCES 



a simple decision tree. In the latter, the student would have to 
back up the tree to some prior starting point in order to come 
down a new path, thereby vitiating the flexibility of this course. 
In the learner-controlled approach there is neither a common 
starting nor ending point. 

In sum, the student taking this course soon realizes that he 
is like a man using a motion picture camera equipped with 
zoom, pan and tilt features. Successive states of magnification of 
the subject expose more and more detail while panning and tilt- 
ing keep the structure and perspective intact at all times. Sym- 
bols attached to each map inform the student of the level of 
"magnification.” 

Results. In order to test the effectiveness of this approach an 
experiment was designed to measure this learner-controlled ap- 
proach against four other experimental conditions of student con- 
trol or the lack thereof. This was accomplished by copying the 
learner-controlled statistics course onto another disk cartridge 
and deleting all the maps and branch options open to the stu- 
dent. In this way a linear statistics course was created which 
was identical in course content with the original version. From 
these two conditions, learner-control (LC) and linear (L), five 
experimental conditions were created, which are summarized in 
Table I. 



Table I 

Experimental Design 



Condition 


Chapter 1 


Chapter 2 


LC - L (n = 10) 


Learner Control (LC) 


Linear (L) 


LC - LC 


LC 


LC 


L - L 


L 


L 


L- LC 
LC* 


L 

LC 


LC 



♦This condition differs from the LC-LC condition in that chapter sequence is not 
a restriction. 

Fifty IBM students served as subjects in this study and 
were divided equally among the five experimental conditions. The 
analysis, of variance model indicated a reliable difference 
(P< .05) between treatments. Tests between individual treatments 
were made with the result that if students drawn from the sub- 
ject population are given the unique freedom (LC condition) to 
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construct a pathway through a segment of elementary statistics, 
on the average they will score higher on the post-test than their 
counterparts under other treatments, save the LC-LC condition. 
In the latter condition, the student learns under conditions of 
learner-control but takes chapters in sequence. 

These results are evidenced in a regression plot as shown in 
Figure 2. The abscissa shows the pretest score (X) for students 
while the ordinate shows post-test performance (Y). The lines 
shown are ones of best fit for each treatment group and are 
found by a least squares criterion. 

□ □□ 

POST-TEST(Y) 




Figure 2. Regression of post-test score on pre-test score for five exper- 
imental conditions. 



□ □□ 

This plot tends to reveal that complete learner-control, with- 
in the study parameters, on the average produced uniformly 
high performance over the entire range of pretest performance. 
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Learning acquired under conditions of anything less than learner- 
control, for this study, produced post-test performance that was 
more heavily dependent on pretest performance. 



A CAI Arithmetic Practice Program 
Peter Dean and Suzanne Sax 

Numerical skill has been the subject of many CAI projects, 
including the first CAI program written in 1958. 

Most of these CAI programs can be characterized as a 
program-controlled algorithm that generates randomly, or other- 
wise, a series of problems for ' 'flash card” presentation to the 
student. Little or no remediation is offered, and, if available, is 
for a prototype problem or class of problems. 

Method . The present CAI program was designed to provide 
elementary school children with number practice problems (addi- 
tion, subtraction, and multiplication) in which they had the 
option of: constructing their own problems, requesting the ma- 
chine to generate them by means of modified random number 
generators, asking the machine to present problems written by 
the author, and seeking specific remediation on problems under 
any of the above conditions. 

The first three options above are labelled e 'instructional 
algorithm” in Figure 3. The program is written as a series of 
subroutines so that the output of one is input to the next rou- 
tine. 

Assuming that the student has decided he will make up 
his own problem, the "problem description” routine permits 
the student to describe his problem in rather free-form fashion. 

The "problem generator” then presents the problem to the 
student in horizontal or vertical format with the appropriate 
arithmetic operator inserted. 

The student’s answer is then checked for correctness by 
another subroutine. 

If he asks for help, a "specific remediation” routine is 
called from a library which proceeds to break down the trouble- 
some problem in a step-by-step manner accompanied by English 
instructions at each step. If the student is still wrong in the 
specific remediation routine, he could be branched to a true drill 
program on basic number facts. 
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□ □ □ 



O 

ERIC 





Figure Flow chart of arithmetic practice program. 



□ □ □ 



The following portions of a student’s interaction are listed 
below: 

(computer) Type a 3 digit number. 

(student) 478 
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ERIC 



(C) 

(S) 

(C) 

(S) 

(C) 

(S) 

(C) 

(S) 

(C) 



(S) 

(c) 

(S) 

(C) 



Type a 2 digit number. 

39 

478 

x39 



6632 



Nooooo! If you wish to try the problem again press 
space bar. If you want me to show you how to do 
the problem, type “h”. 



h 



Multiply the units digit on top, 8, by the units digit 
of the multiplier, 9. 

478 
x 9 



Multiply the next digit on top, 7, by the units digit 
of the multiplier, 9, and ad^the tens digit, 7, to 
the product. 

7 

478 
x 9 



02 



M'ove the tens partial product one place value to 
the left and sum the partial products. 

4302 
+ 1434 



18642 



And you have solved the problem! 



Results. Experiment,:', results from 120 grade schoolchildren 
on an earlier version of this course (lackirg the specific remedia- 
tion feature) suggested that merely giving control to the student 
did not produce significant group post-test gains, compared with 
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linear controls. Rather, specific remediation is also needed and 
is now added to the course as illustrated above. The results 
from this additional feature are not yet known. 

One finding from the study disclosed that the use of learner- 
control with 6th graders resulted in significant savings in prac- 
tice time. 



Decomposition of Numbers into Primes 
Pat Beatts 

This program segment was written in a tutorial mode to 
give instruction in breaking down numnevs into primes. Note 
the following free interchange between this student and the com- 
puter: 

(c) O.K., to clarify the rules for decomposition by the 

first method, you must answer 5 questions. Ready? 
Hit the spacebar. 

L What do we divide by first? 

(s) 2 

(c) 2^ When do we stop dividing by this? 

(s) When we get to 39 

(c) Are you sure you are ready to do this? Anyway try 

once more. 

(s) When the product will no longer divide by 2 

(c) Fine 

3^ What do we divide by next? 

(s) 3 

(c) 4^ When do we stop dividing by this? 

(s) When 3 will no longer go into it 

(c) Good! One last question. 
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5^ When do we stop trying to divide any further? 

(s) When we have all prime numbers 

(c) Now that you’ve finished, let’s look at the whole 

of the 5 questions and the correct answers all 
together. 



Introduction to Computing 
Suzanne Sax , Ralph Grubb and Peter Dean 

How a computer works remains a great mystery to many 
people, including large numbers of employees in the computing 
industry. Various attempts to remedy this have included class- 
room exercises which make elaborate use of magnetic chalk 
board displays, students serving as signaler, accumulator, etc. 
While each method has had some degree of success, none, I 
think, has the promise found in the present CAI approach. 

In this program a student can type on his keyboard a sim- 
ple arithmetic expression of his choice. This can be as simple 
as 2 + 3 or more advanced, as (((24 - 4 - 3) x 5) +4). Rather 
than providing a direct numeric answer, this program translates 
the expression into English and assembles it into a quasi- 
flowchart form for display on the CRT (see Figure 4). 

At this point in the process the student has already been 
introduced, perhaps without his awareness, to several levels of 
coding inherent in computing. The student now has the option 
of seeing a mini-computer execute the results. 

This simulated computer’s internals include: thirty locations 
of storage, an I/E time indicator, instruction counter, address 
register, data register, instruction register and accumulator— all 
of which are displayed on the student’s CRT. With the original 
arithmetic expression still in view, he now sees his data as en- 
tries in storage locations in addition to seeing a program that 
has been built to execute the problem. The program instruction 
set is composed of System/360 mnemonics, i.e., 1 — load, a = 
add, d = divide, etc., (see Figure 5). 

The student can now execute the program in a single-step 
mode, if he wishes, by repeatedly pressing the space bar on his 
terminal and observing the data flow within the simulated com- 
puter > 
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□ □ □ 




Figure 4. CRT display of program steps necessary to evaluate a speci- 
fic arithmetic expression. 

□ □ □ 

At any point he can branch out of the procedure to: parti- 
cipate in the register displays by filling in the expected contents 
from keyboard responses; go back to the flow chart segment and 
construct another arithmetic expression; or write a program of 
his own and have it executed! 

RESOURCES NEEDED 

Without going into any great detail, let me quickly list the 
resources needed at this time if a CAI effort is to be successful. 
Regardless of the size of a research and development project in 
CAI, certain functional skills are required. These skills are gen- 
erally the result of backgrounds in educational psychology, ex- 
perimental design and statistics, author-teacher experience, 
instructional programming, instructional coding, familiarity with 
computer assembly languages, cardpunch operation, computer 
operation, administrative co-ordination, supervision and support, 
in addition to teaching— and learning from— students. 

In rare instances, a small group of people may possess all 
these functional characteristics. Usually a larger team is needed 
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□ □ □ 




Figure 5. CRT display of simulated mini-computer executing the pro- 
gram steps shown in Figure 4. 



□ □□ 



in order to put the project on a reasonably large enough scale to 
meet realistic time schedules and make efficient use of the sys- 
tem. This would tend to indicate that, when a project of such 
scope is launched, the size of staff should be adequate to handle 
a ff stable ' 9 of several subject-matter experts. In this way, full 
use of all talents and the hardware is most likely. 

This leads to the inescapable conclusion that, with a large 
computer system and the time and talents of diverse experts, the 
originator of the course-content must have sufficient ability, 
talent, and expertise in his field to warrant the expenditure of 
time and effort. The pay-off is the wide adoption of the finished 
CAI course. 

TRENDS 

A decade of CAI experience lies behind us. It is timely and 
appropriate to look for substance. What have we learned? What 
trends in CAI are emerging? What is needed at this time in 
terms of resources? 
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It will be remembered that CAI is the use of a computer to 
improve the student’s interaction with his subject matter, ma- 
terials and teachers. The following views, entirely personal, will 
be tied to that definition by identifying trends and needs with 
such concerns as the student, modes of interaction, subject matter, 
materials, and teachers. The list will be suggestive rather than 
exhaustive. 

The student. From the illustrations in this discussion and 
elsewhere, I predict a greater emphasis will be placed on learner- 
control in CAI. In addition to the cursory arguments mentioned 
earlier, researchers will find that the students’ audit trails through 
the subject matter, as supplied by the computer, will be invalu- 
able in discovering the nature of what is now referred to as 
"cognitive styles.” 

New developments in natural language processing in Course- 
writer will also permit the student tar greater freedom to inter- 
rogate data bases in CAI. 

Modes of interaction. Because learners will assume more 
control in CAI programs, there will be greater utilization of 
such techniques as exploration, gaming, simulation, etc. 

Subject matter . Granting more freedom to learners, coupled 
with natural language facilities, will cause narrow course boun- 
daries to drop away; narrowly delimited courses will be replaced 
by a broad content area or knowledge space whose dimensionality 
will be found in the process. 

Materials . The increasing use of multiple modes of interac- 
tion will suggest greater use of materials resource centers in con- 
trast to building an entire course on a programmed text or CAI 
terminal. This in turn raises such questions as, "What objectives 
might best be met by the student, the instructor, library, lab- 
oratory, text, teaching machine, computer, etc.?” That is to 
say, education has been, and should continue to be, a series of 
experiences, both direct and mediated. 

If the earlier predictions are realized, CAI terminals will 
need to be richer. Graphic displays, light probes and other 
associated audiovisual components will be needed if one is to do 
more than automate programmed texts. 

The teacher. We know by definition that the teacher will be 
an integral part of CAI; however, studies conducted with pro- 
grammed instruction have shown that it requires better teachers 
to teach in a context of automated instruction. This is under- 
standable when one considers the larger role the teacher plays, 
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which includes offering special help to students, each at a differ- 
ent place in a program. Furthermore, CAI is still a foreign and 
specialized topic to many educators. These factors suggest that 
training centers must be established in order to prepare teachers 
for these functions. 

The author languages used for CAI courses also deserve 
attention since they still bear close resemblance to computer 
coding languages. 

A recent development by Peter Dean, Ralph Grubb and 
Mike Dowsey has resulted in a course planning form and a 
computer preprocessor program that makes for a simplified 
author entry system which is machine and language indepen- 
dent, self-documenting, faster, and requires less author training 
than previous systems for many types of CAI authoring. 

Since the major cost of CAI lies in program preparation, 
developments in this area could greatly influence the economics 
of CAI and ultimately its acceptance. 

A final comment I might make with respect to the learner- 
controlled or exploratory mode of CAI is that despite the ad- 
vances here described and currently underway, it will require 
continuing research to evaluate this new approach as a means of 
permitting unique freedom of exploration to any student within 
a subject. It seems likely that some other subject, less rigorously 
constructed than statistics, might be a better vehicle for evaluat- 
ing the exploratory mode. In this connection, we might speculate 
on the possibilities inherent in a course such as English Restora- 
tion Poetry, or the Development of the English Novel. If this 
suggestion seems farfetched, consider the following quotation. 



The owner of this magic desk will, in fact, have at 
his disposal in compact form what v. ^uld nowadays 
be the whole contents of a colossal reference library 
of millions of volumes. Moreover, by a system of in- 
dexing and automatic cross-reference, based on elec- 
tronic selection, he will be able to extract what he 
needs by pressing a few keys, instead of going 
through the long process that is now necessary in 
public libraries.* 



♦Earl Mountbatten, President of the British Computer Society, in his presidential 
speech to the Institute of Electronic and Radio Engineers, 1946. 
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In spite of my title on the program, I do not have a great 
deal to say about networks except as they relate to a broad con- 
cept of sharing facilities which is one means of maximizing our 
future efforts. What I will try to do is, first, give you an idea— 
particularly those of you who have not had the pleasure or pain 
of dealing with it— of what you might expect from the usual 
University Computing Center; second, say something about 
special computing needs as I have seen them in a medical center; 
third, discuss the possibilities of sharing facilities and skills with- 
in a campus, as well as similar sharing between separate schools; 
fourth, make some remarks on the development of computer aids 
to learning, and give examples of same application areas for 
conversational use of the computer with relevance to professional 
education; and finally, show where some improved man-machine 
communication methods can take us. That is probably enough 
for a day or two, but I will try to do something with that list 
in the next hour. 

FUNCTIONS OF A UNIVERSITY COMPUTING CENTER 
Now, how can you make use of a University Computing 
Center and avoid being trapped, like I was, into running it? 
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More and more, it seems possible that such centers may become 
versatile enough so that you may not have to. This did not 
seem true to me when I originally became involved and, as I 
say, it may be considered a kind of trap, which I think most 
of you should work hard to avoid. 

Most academic computer centers have grown to fill not only 
their original scientific needs but also to provide computing ser- 
vices to research investigators who themselves have skills in 
mathematical or statistical analysis. The needs of administrative 
data processing have almost always been considered as separate 
problems and facilities for meeting such needs have developed 
independently. It is still true that the requirements of these two 
areas are quite different and that the backgrounds and methods 
of programmers who work on these different kinds of tasks have 
completely different life-styles. 

In the setting of medical education, however, there is joint 
use of and joint concern with patient records required for the 
basic purposes of the institution which must also serve the needs 
of patient care and instruction of students in clinical and related 
matters. Interest in patient records is shared by administrators 
and managers concerned with the operation of the hospital, by 
instructors and students who are studying the clinical material, 
and also by research investigators who would like to make a 
different use of the clinical data. The daily operational concerns 
of those involved in these three factions of a university medical 
center are quite different and one can hear many predictions of 
failure when attempts are made to integrate services for them. 
The scheduling demands are different, as are the needs for dif- 
ferent facilities and different kinds of supporting staff. 

Some of the special needs of a medical computing center as 
against a general university campus computing center are a result 
of this close relationship between administrative activities and 
instructional and research activities. We somehow have to take 
care of the business aspects of running a hospital and the 
clinics. We have to build methods of handling patient record in- 
formation. We need data acquisition and control of other inter- 
connected apparatus, such as auto-analysers in the clinical labo- 
ratory, and things of this kind. 

We have other administrative needs which may differ from 
a general campus. Operations within a medical library are usu- 
ally quite different from those on a general campus. For example, 
there is a much heavier demand for serials and periodicals in 
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the total operation than is usual elsewhere. Concerns about ad- 
missions to professional schools and student information in gen- 
eral are likely to be quite different. The scheduling of very high- 
cost facilities for interlocking kinds of professional curricula also 
introduces different concerns. 

It might be of some interest to give a brief listing of the 
kinds of administrative projects currently underway in our setting 
which give a hint of what areas have sufficient pressure on 
them to get attention first. Outside of the hospital, we have 
projects such as a registration subsystem of a student record 
system; a revision, completion and maintenance of a medical 
school admissions system; a contracts and grants accounting 
system; a library serials system; grounds and buildings project- 
control and inventory; an academic personnel system; a person- 
nel skills-locator system for transfers and promotions in the per- 
sonnel department; a courses and grades subsystem of the student 
records system; a space utilization system, and so on. 

Within the hospital itself, we have an equally” demanding 
list of projects including modifications of the hospital business 
systems which, with all the interacting insurance plans, Medi- 
care* and things of this sort, get to be very complex operations. 
In our hospital, for example, approximately six thousand bills 
need to be sent out each month. The system has projected into 
three separate phases with a great deal of detailed planning in 
each one. 

Further projects involve the development of a clinical labo- 
ratory test-result reporting system; development of a medical 
records index, which is just the very first pass at the medical 
records— in an attempt to help people locate charts; and a re- 
vised census and bed-control system, so that people know in a 
timely fashion which beds are available and which patients are 
in which beds. 

Let me just mention once again some of the possible advan- 
tages of integrated operation between these kinds of concerns— 
research and instructional concerns — which may somewhat off- 
set the difficulty of trying to run these things jointly. There is 
a possibility of sharing file-access. I have already mentioned the 
kinds of overlapping files that might be possible. You could in- 
deed make these people, with different points of view and differ- 
ent concerns, reasonable bedfellows. Further possibilities include 
sharing of staff skills and knowledge about what is available in 
a large computing facility; the sharing of programming systems; 
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and the possibility— which I heard a previous speaker mention — 
of a useful mix of instructional interactions and file-access, for 
example, where methods developed in one unit may very well 
have some application and usefulness for another. 

One concern in the medical center which is very difficult 
to handle in a truly centralized fashion is data collection, con- 
trol of apparatus, and control of experimental manipulations. 
Here I think the sensible route is the combining of local proces- 
sors, which do this kind of immediate handling of data, and 
some linkage to a central computer that becomes useful either 
for a large analysis, large processing needs, or for access to the 
large central files. 

An important but secondary advantage of combining local 
processors in one facility is that joint, and sometimes overlap- 
ping, justification for additions to the facility becomes possible. 
Some portion of the machinery, for example, may be justified 
by one program much more than by another, and yet the second 
program may very well benefit from that machinery having been 
installed. The management and administrative uses are the ones 
which push heavily in the direction of large, random-access file 
capacity. Although it might not be possible to justify such a 
huge capability for research or instructional purposes, it never- 
theless becomes very useful to them to have that capability avail- 
able. Instructional requirements may push very heavily for inter- 
actional use of the computer from remote points and this in 
turn may be especially useful, although not as easy to justify, 
for informational efforts of the administrative people, or the 
research people. There have been very few attempts to develop 
medical center systems which really try to handle this problem 
on a centralized bais. One of the few is the ACME system at 
Stanford which does not try to handle a lot of the administrative 
tasks that I have described to you, but which is relatively suc- 
cessful in centrally handling the data-acquisition control problem. 



INSTRUCTIONAL FACILITIES - HARDWARE 
AND SOFTWARE 

At the San Francisco Medical Center of the University of 
California, we have organized an office with divisions entitled; 
Management Systems, Instructional Systems, Research Systems, 
and the Computer Service Facility. I do not think there is any- 
thing very remarkable about that because the activities of teach- 
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ing, research and administration are the natural characteristics 
of university life. 

I am no longer going to talk about either the mangement, 
administrative area or the research area, which I think you are 
probably well able to imagine and to find in one form or another 
at different places, but instead J will give you a little detail on 
the direction of development within the instructional division, 
since that is the major interest of this conference. 

Our computer system is a mix of facilities which is partially 
justified in various ways by the needs of the separate portions 
of the operation. We have an IBM 360 Model 50 with 256,000 
bytes of high-speed core, a million bytes of slower speed core, 
and a 2314 disk unit with eight drives at present and four more 
to be added this spring. We operate a system that provides two 
batch partitions of 140,000 bytes of storage each, which can be 
combined for large-size jobs by a simple control on the job 
card. Because of the large disk-storage capabilities, a wide range 
of library programs are accessible on call. 

There is a remote job-entry access to these batch operations. 
One batch terminal, for example, with its own printer-reader, is 
located at a junior college about 50 miles from our center. Re- 
mote entry for batch work can also be accomplished via type- 
writer terminals, such as teletypes or 2741 terminals. We are 
operating four teleprocessing systems: CPS, which is an interac- 
tive subset of PL/ 1; APL, which is a mathematically oriented 
interactive programming language; and ATS, which is a kind of 
secretarial aid for storing, editing and retrieving text information. 
It is very useful for specifications and proposals which are 
changed in some degree from one typing to the next. 

The fourth, teleprocessing system is an interactive conversa- 
tional programming language called PILOT— Programmed In- 
quiry Learning Or Teaching— which is a system that we develop- 
ed.* It has many similarities to other CAI author-languages, but 
we feel that it also has advantages over others. 

The instructional division of this office that I have been 
describing has been working on this improved language for ex- 
pressing conversational strategy. Until very recently, it has been 
centering most of its efforts on that project, and also on the 



* Starkweather, J.A., PILOT and the preparation of program-oriented medical records 
for computer-assisted learning. In L.M. Stolurow et al. (eds.) Computer Assisted In - 
struction in the Health Professions , Newburyport, Mass., Entelek, 1970. 
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production of brief demonstration programs for the purpose of 
discovering those professional faculty members in specific con- 
tent areas who are both interested and excited about the potential 
of computer assistance to learning as applied to their specialty. 

There is a need for this group to develop both marketing 
and consultation skills. If the group were able to expand to any 
extent, instructional division plans might involve a subdivision 
into specialized subsections. One would manage elements relating 
to development, maintenance, and improvement of the language 
system itself. Another subsection would have to do with training 
people to make use of that language and helping instructors 
program curriculum elements in it, plus documentation of such 
efforts. Perhaps a third subgroup would act as liaison with users, 
sending information on local and remote operations to a mailing 
list of people interested in what wc are up to. Another subgroup 
might handle the development of conversions from curriculum 
materials written in the other author-languages in other places. 

I will mention this point again a little later. 

DEVELOPING INSTRUCTIONAL MATERIALS 

At this time, let me make a few remarks about the instruc- 
tional group and how its system tries to meet some of the needs 
of students and teachers. This will not be the first point in 
these meetings where the notion and importance of corrective 
feedback in the development of interactive instructional materials 
will be expressed. Corrective feedback can be a powerful mecha- 
nism in the control of any dynamic process and is especially 
valuable in the development of new procedures which can not 
be completely predicted in advance of their operation. I think 
that a good deal of what we do in using the computer for in- 
struction falls in that category. 

In developing a programmed instruction sequence between 
man and machine, we must % be particularly cautious when set- 
tling on a method which seems to work well in one instance. 
For these conversational operations, a change in context or set- 
ting seems to have a powerful effect on the interaction, some- 
times with quite disruptive results. This concern can be addres- 
sed, I think, by arranging that such systems of man-machine 
communication and interaction have a mechanism by which the 
user can record comments about its handling of his responses or 
remarks on its occasional malfunction. In other words, I think it 
is important that there be a gripe mechanism for the user of 
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the system, whether student or researcher. Comments should be 
put to use as rapidly as possible, with a subsequent improve- 
mer* in succeeding interactions. The system itself ought to make 
such revisions as easy as possible. 

A developer of such systems should leave the curriculum 
content in the hands of professionals in the specific subject-matter 
area. But he should be in a position to provide such people with 
methods by which they can receive corrective feedback informa- 
tion from students and easily review such feedback, taking cor- 
rective action. 

It is possible and usual for human instructors to make use 
of centrally produced, standard textbooks and other curriculum 
aids. It is also possible for them to use textbooks— standard, 
nationally-distributed aids to the instructional area— and inter- 
pret such materials to fit a local context, assisting the student in 
understanding them. 

It seems to me that it is exceedingly difficult, maybe even 
impossible, for computer programs — at least in their present stage 
of development— to accomplish this same kind of task. We should, 
therefore, not expect to produce centralized, standard curriculum 
materials for computer presentation except in very basic areas 
of routine drill. I think that we can try to produce examples 
and points of departure for a local instructor by providing these 
centrally produced materials, but the instructor should be in a 
good position to test them and modify them easily to meet the 
needs of the local context, customs and language usage. 

There is an interesting version of this problem which occurs 
in arranging for a computer program to recognize and understand 
the language produced by a student or other user in a conver- 
sational situation. Let us assume that the author of such a pro- 
gram does have versatile mechanisms at hand with which to 
describe language elements. Even so, he cannot predict the entire 
range of possible responses to a question which does not severely 
limit the format of the reply. If the situation is such that the 
author can expect feedback from test subjects who face his pro- 
gram, then it is sufficient for him to write only an initial 
skeleton which he will expect to fail on first attempts. Informa- 
tion from users will allow him to quickly add other elements of 
recognition, which will rapidly improve the program. 

The rate of such improvement is clearly a diminishing 
kind of operation. There is a very rapid improvement in this 
respect in the first trials, as one gets the first feedback from 
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early subjects. After that, however, it becomes less and less a 
part of the problem. One can always come across a new type 
of respondent, perhaps a poet who uses some language which the 
previous people had not thought about using, but this will be- 
come a very rare occurrence. 

One can imagine improving this process in a way which 
might make program development much less painful from the 
author’s standpoint. Future versions of an author-language 
might have provisions for an active monitor or prompter termi- 
nal, where an author/ instructor would sit and converse with a 
student who sits at a separate terminal. It is conceivable that a 
program could be written not only to record appropriate portions 
of this interaction but also to abstract, from the spontaneous 
responses of the author, elements necessary for the construction 
of automatic reply. This is probably a very big order, but a 
little easier for me to imagine than a completely automatic lesson- 
generator program. Here we are talking about a pattern recognizer 
of sufficient capability to abstract elements from humanly gener- 
ated material. I think that this is, at least, something worth 
thinking about and working on. 

On successive runs with new students, the program would 
first attempt to recognize a student’s reply by virtue of previously 
recorded elements. In the event of failure, it would indicate at 
the monitor terminal that the author should insert a new human 
response. When this kind of thing occurred, the author would 
immediately come on the line with a response that he would 
generate. An interactive process of this sort might result in the 
building of a functioning program which would progressively 
handle more and mote responses. Before long, the author could 
step aside, I would hope. 

If a program were to be transplanted and used in a context 
where the language characteristics of students were different from 
those in its original location, a similar process would have to 
be undertaken in order to bring the program into line with the 
new setting. For example, it is clear that the language back- 
ground of students in central-city schools is likely to be quite 
different from that found in suburban districts. A computer pro- 
gram which recognizes the language in one setting will quite 
likely fail in another. The required process of translation for 
such a program may result in new knowledge about language 
habits in the two environments. So there is a possiblity of some 
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useful fallout from this kind of interchange and rearrangement 
of material.. 

Let us think for a moment about the requirements for 
language-system handling of conversational interaction. I have 
already suggested that a language for writing conversational inter- 
action, built to recognize appropriate elements of naturally occur- 
ring language, should have both a high level of readability and 
an editing method which makes it as easy as possible for a 
person who has curriculum concerns at the local level to make 
changes to meet those needs. That person should have a means 
to record comments about language operation and make use of 
those comments to change and improve the program. 

It would be especially valuable for such a system to have 
a subset of mechanisms for simple operation and easy entry to 
its use. With such mechanisms, a local teacher or curriculum 
coordinator could make use of the language with little effort. We 
expect that the system should itself instruct new users in the 
basic aspects and initial operations that would be necessary. The 
system should be capable of handling and storing text in a very 
flexible fashion and at the same time should make use of rapid 
recognition methods in tracking the meaning of the subjects’ 
responses. 

Basic recognition methods should be sufficiently rapid so 
that the pace of conversational interaction is not badly distorted. 
While it should be possible to designate a specific sequence of 
programmed responses to subjects, there also exists a need for 
random choice mechanisms so that an author can call on a 
varied output. Future systems should probably have the develop- 
ment of adaptive mechanisms to improve their responses as a 
result of increased experience with many subjects. 

There are already a variety of what might be called CAI 
author-languages. While we feel there are many advantages to 
the approach we have taken with PILOT, we recognize that we 
have spent a lot of time in working on this system. In the 
meantime, there has been a considerable amount of effort ex- 
pended elsewhere in the development of interesting and useful 
curriculum materials in other languages. So we are interested in 
translation from one language of this type to another. This does 
not seem to be an insurmountable problem. 

I believe that there are many potential advantages to shar- 
ing between centers. First of all, one can conceive of transplant- 
ing the whole system from one place to another, that is, what 
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usually amounts to new implementation of the language on a 
new machine. This can be a difficult process, depending to some 
large degree on how the language system is built. In our case 
we have tried to make PILOT relatively readable by writing the 
system itself in a high-level language, and we have had some 
success in implementation on other machines. The most notable 
success is a translation to the SDS 940 time-sharing system. In 
this instance, it was a manual translation, but the process went 
amazingly rapidly, in comparison with other projects of similar 
type, probably because of the fact that the original system was 
written in a high-level language. 

I have already mentioned that the sharing of curriculum 
elements is likely to require translation from one author-language 
to another. I believe that experimentation in this area is quite 
valuable because of the great advances that could accrue from 
the transplant of curriculum materials from one professional 
school to another. A particular advantage in such sharing is 
the gathering of evaluative data on such programs. Larry Stolu- 
row was discussing with me ways to improve the evaluation 
process— -to reduce those extensive delays which occur when 
materials are evaluated at only one school. After the first group 
or class of students tries a program there may be a lapse of as 
much as a year before another appropriate class becomes avail- 
able for further evaluation. It would be extremely valuable, I 
think, to try those same programs in a number of settings and 
improve things much more rapidly. 

There is also the possiblity of joint and reciprocal develop- 
ment of curricula, i.e., people at separate centers might combine 
their efforts in the development of the curriculum itself. It would 
be an interesting experiment to bring together specialists in adja- 
cent areas or in the same area, with different points of view 
and varying abilities to collaborate, as the joint authors of a 
textbook might. In this instance I can imagine them interacting 
with the same system, perhaps even calling in to the same 
machine from different centers and reporting the development of 
materials and their trials of them. 



EDUCATIONAL NETWORKS 

Let me turn from this to what I want to say briefly about 
networks. I conceive of educational networks as being a continu- 
um of possibilities, if the definition is kept rather loose. The 
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full-scale network might be considered the direct interconnection 
of machines at different campuses where actual sharing of com- 
puter load could take place. If one place became overloaded, data 
would be shipped to a remote point for processing. There are 
efforts underway along this line, primarily those supported by 
the Department of Defense and its collection of time-sharing 
operations at various high-powered centers around the country. 

A second level might be the reciprocal remote usage of 
time-sharing computers by using an appropriate terminal to call 
in from the remote points. This is a network only by virtue of 
the fact that the telephone company is a network. By making 
use of that existing network, it is certainly conceivable that edu- 
cational institutions could do more and more of this kind of 
network activity. Someone who wanted to try a system that 
existed uniquely at our center could arrange to call in from an 
appropriate terminal anywhere, try the system out, and decide 
whether they want to try to transplant it or not. We might find 
that there is a program that we would like a specialist in one 
curriculum-area of our professional school to experience. The 
program exists at the Harvard Medical Center, let us say. We 
would arrange for our specialist to try it out remotely and would 
simply pay the long distance charges, obviously a much lesser 
expense than sending him to Harvard or making a full-scale 
attempt to transplant the system. That kind of network arrange- 
ment may prove to be especially useful. 

A third level is the sharing and translating of programs 
mailed from one place to another. I think, in that sense, we are 
simply considering the mail system as the network mechanism. 
There are attempts underway to develop a kind of clearinghouse 
of information about available programs which would make this 
kind of interaction more useful and generally available. An enter- 
prise called EDUCOM, which has members from many universi- 
ties in the country, has throughout its existence been interested 
in networks at all the levels I have just described. This group 
was the primary author of a book describing full-scale networks 
of the first type. They have made proposals for the development 
of networks of the second type and they are currently trying to 
implement a network of the third type by acting as a clearing- 
house of programs between university centers. 

EVALUATION CRITERIA 

Now let me turn back to the instructional area and make a 
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few remarks about problems in evaluation and development of 
criteria for programs of this kind. Anyone who sets out to 
improve instructional methods soon realizes that objectives and 
. ' criteria for successful instruction are seldom expressed in terms 
that are specific enough to be at all useful for measurement and 
evaluation of instructional methods and processes. When the 
whole system is an experimental one, such as the various means 
by which the computer is involved in learning and instruction, 
a specific listing of objectives and criteria of performance becomes 
especially valuable. This kind of information can be fed back to 
a program author, who may be in a position to compare more 
than one method of presentation of the curriculum materials. If 
the criteria are sufficiently objective and measurable, then statis- 
tical techniques such as discriminant function analysis or factor 
analysis might be used to discover which items of input are 
especially relevant to the outcome. All of these things require a 
good criterion. 

I suppose it is a platitude to say that these objectives should 
be stated in terms of the observable behavior expected of a stu- 
dent at the time that he completes the sequence. I think that 
when this notion is not considered, we have many elements in 
professional education which go badly awry. 

Students are often asked, for example, to give a list of the 
signs and symptoms associated with a disease. At certain points 
in the curriculum, students get much less practice in attempting 
to solve a diagnostic problem on the basis of presented symtoms, 
which is the reverse kind of information. I think that in this 
particular area interactive instruction can indeed allow a student 
to practice behavior which is closer to his eventual goal. 

EXPLORATORY USES OF CONVERSATIONAL 
INTERACTION 

I am now going to present some exploratory examples of 
the variety of CAI conversational interactions relevant to profes- 
sional education. We have had a special interest in allowing and 
handling unconstrained responses and this has of course led us 
into considerable exploration of the problems of language- 
handling involved in this mode of operation. There is a whole 
continuum of possiblities, ranging from the extreme, constrained 
format of presenting a statement or a question and asking for a 
yes-or-no response, through the possibility of multiple-choice 
responses, fill-in responses, and completion responses, to the 
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seemingly free-form response which, however, has subtle con- 
straints that are very hard to pin down. 

But we have tried to be as free as possible in our develop- 
ments. When our language pattern-recognition methods and our 
processing methods failed us, and we did not recognize what the 
person was talking about in his response to the program, our 
usual recourse was to fall back on a greater restraint, with a 
more solid boundary on the context. The extreme case of this 
is, "Would you answer yes or no to the following: . . I 
think you will find cases of this sort in the examples to be 
shown. 

In trying to develop this kind of language-handling capabil- 
ity, one of our early attempts, for test purposes, was to use the 
kind of verbal questions used in the individual administration 
of an intelligence test. 

Example 1 shows the kind of simple-minded early approach 
which let us play with the variety of responses that a person 
might give to this kind of question. We encouraged the people 
responding to these questions to make every attempt to beat the 
> system, so that we had lots of people trying to play pranks and 
give partial responses or to give responses that would have to be 
considered correct but had not been anticipated. 

EXAMPLE 1 

WHAT STATE ARE WE IN? 

. I think we are In California 

RIGHT, AND ON THE FIRST TRY. 

WHO WAS THE FIRST PRESIDENT OF THE UNITED STATES? 

. George 

YOU MUST KNOW MORE THAN JUST THE FIRST NAME . . . 
WHAT IS HIS LAST? 

. Washington 

NOW YOU’VE GOT IT. 

HOW MANY PINTS MAKE A QUART? 
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. two 

THAT’S THE WAY! 

WHAT IS THE CAPITAL OF ITALY? 

. Is it the Vatican? 

NO, IT’S NOT THE VATICAN, CHOOSE ONE OF THESE . . . 
MADRID ALGIERS VIENNA LISBON ROME ATHENS 

. Athens 

YOU SHOULD LOOK THAT ONE UP. 

Notice that the remarks printed entirely in capitals and in- 
dented are those printed by the computer. In this system, the 
typing ball returns to the left of the page and prints a period, 
which indicates to the user that he is free to make a response. 

A similarity question, such as that in Example 2, is one 
which can have a fairly free response and therefore has to have 
a fairly complicated background of possiblities, both at a concrete 
and at a more abstract level, to make appropriate remarks about 
the response. 

EXAMPLE 2 

THIS NEXT SECTION HAS QUESTIONS ON THE SIMILARITY 
OF THINGS. 

HOW IS AN APPLE LIKE AN ORANGE? 

. They are both round. 

THAT IS ACCEPTABLE, BUT TRY TO GIVE A MORE BASIC 
SIMILARITY. 

. Both of them grow on trees. 

TRY ONCE MORE TO THINK OF SOMETHING MORE BASIC 
OR GENERAL. 

. I don't know any more. 
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YOU COULD SAY THAT THEY ARE BOTH FRUIT. 

A proverb (Example 3) is an even worse case, in which 
the response can be very abstract and still carry some of the 
meaning appropriate to an interpretation of the proverb. In this 
example, the response, "They might break the glass,” would be 
considered a very concrete, non -explanatory kind of response to 
the question. 

EXAMPLE 3 



NOW I WOULD LIKE TO ASK YOU ABOUT SOME PROVERBS . . . 

SEE IF YOU CAN TELL ME THE GENERAL MEANING OF 
THESE PROVERBS, OR WHAT A PERSON WOULD MEAN IF HE 
SAID THEM TO YOU . . . 

FIRST TRY THIS ONE. . . 

A BIRD IN THE HAND IS WORTH TWO IN THE BUSH. 

.Well, you can be sure of having the one in your hand for dinner. 

YOU HAVE THAT MEANING IN HAND. HERE IS ANOTHER . . . 

PEOPLE WHO LIVE IN GLASS HOUSES SHOULD NEVER 
THROW STONES . . . 

. They might break the glass. 

I GUESS YOU COULD INTERPRET IT THAT WAY, BUT TRY TO 
BE MORE GENERAL IN YOUR ANSWERS. 

Example 4 shows a fill-in technique, which makes things a 
little easier. The previous example was of the more difficult type. 

EXAMPLE 4 

NOW I WILL ASK YOU TO FILL IN SOME BLANKS ABOUT 
PROVERBS . . . 

THIS PROVERB SAYS IT IS NO USE TO CRY OVER SPILT MILK. 
HERE, “SPILT MILK” COULD REFER TO ANY 

(TYPE ONE WORD, PRESS RETURN) 
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. drink 

NOT WHAT I HAD IN MIND — TRY IT THIS WAY . . . 

THIS PROVERB TELLS YOU NOT TO ABOUT 

SOMETHING THAT IS • 

(TYPE TWO WORDS, THEN PRESS RETURN) 

. worry spilled 

YOU HAD ONE OF THEM RIGHT — LET’S GO ON. 

Example 5 shows another situation in which you can imi- 
tate a picture arrangement sequence or something of this sort. 
This kind of thing is often done with a sequence of cartoon 
frames, putting them in an appropriate order to tell a logical 

story. 



EXAMPLE 5 

I WILL SHOW YOU SOME SETS OF STATEMENTS. ARRANGE 
THEM SO THEY TELL A LOGICAL STORY, THEN SHOW ME 
YOUR ARRANGEMENT BY TYPING THE CORRESPONDING 
LETTERS. 

A. BATTER HITS BALL. 

B. BATTER SCORES HOME RUN. 

C. PITCHER THROWS TO BATTER. 

(TYPE THREE LETTERS, THEN PRESS RETURN KEY) 
.CAB 

FINE, HERE IS ANOTHER .. . 

A. GANGSTER KILLS MAN. 

B. MAN IS FOLLOWED BY GANGSTER. 

C. MAN WALKS DOWN STREET. 

NOW TYPE THE SEQUENCE OF LETTERS, PRESS RETURN. 
.ABC 

NO, DEAD MAN CAN’T WALK. 
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While not a difficult method, from the computer-processing 
point of view, Example 6 demonstrates a reasonably difficult 
problem— and is one which is a good deal more difficult if 
your information is limited to just the listing of items A through 
G. It takes a bit of thought to put those elements together. This 
illustrates the possibility of a prompt and a re-try with some 
additional information added. 



EXAMPLE 6 



HERE IS ONE A LITTLE MORE COMPLICATED . . . 

A. THE HORSE WAS STAGGERING UNDER HIS LOAD. 

B. HIS DONKEY WAS LOADED WITH SACKS OF SALT WHILE 
HIS HORSE WAS LOADED WITH SACKS OF SPONGES. 

C. HIS HORSE TROTTED ALONG EASILY. 

D. A MAN WAS TRAVELLING WITH A DONKEY AND A HORSE. 

E. THE DONKEY TROTTED ALONG EASILY. 

F. HIS DONKEY WAS STAGGERING ALONG UNDER THE LOAD. 

G. THEN THEY HAD TO CROSS A STREAM, AND SWAM THRU 
DEEP WATER. 

. EADBG FC 

I DON’T THINK YOU HAVE THE IDEA. 

TYPE YES IF YOU WOULD LIKE TO HAVE A HINT AND TRY 
IT AGAIN. 



. yes 



***THINK OF, WHAT HAPPENS TO THE SALT AND*** 

* TO THE SPONGES WHEN THEY GET IN THE WATER* 

•DBCFGAE 

FINE, THAT WAS ONE OF THE ARRANGEMENTS I HAD 
IN MIND. 

There have been many approaches to the use of the com- 
puter for content analysis of language material.* I would like to 



♦Starkweather, J. A., Overview: Computer-aided approaches to content recognition. 
In G. Gerbner et al. (eds.) The Analysis of Communication Content . New York: 
John Wiley & Sons, 1969, 339-342. 
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make just a few remarks about that area of research. It is possi- 
ble to arrange computer approaches to content analysis on the 
dimension of the importance of having an immediate response. 
At the extreme of leisurely response we might find research 
which is involved with the analysis of many different indivi- 
duals whether they be speakers or writers, in different situations. 
Included here are attempts to analyze written content in order to 
decide whether Jefferson or Madison wrote a specific paper or 
whether Shakespeare or Bacon wrote a particular play. The idea 
is to try to develop measures of the formal characteristics of 
language usage which would indicate things of this sort. 

At the extreme of immediate response we have something 
like what we are dealing with here: a need to cope with a par- 
ticular point in an immediate conversation with a specific per- 
son. In the first area, content analysis of documents and similar 
investigations of this type, the investigator usually deals with 
standard units of writing or speech, i.e., words or phrases 
which might suggest anxiety, words that suggest anger and so 
on. We could classify collections of terms depending upon the 
interest of the investigator. We assume that by studying the 
relative frequency with which such units occur, we can produce 
a new and useful description of a document or a transcript of a 
speech. One such program is called the General Inquirer. Phillip 
Stone from Harvard is the author. This program expects incom- 
ing textual data to be arranged as sentences and it examines 
the text by making references to a dictionary of word stems— a 
huge dictionary— in order to handle virtually all terms that 
might be included. The approach, then, involves an effort to 
produce a dictionary which, while specialized in coverage, never- 
theless attempts to handle the vocabulary usage of different 
speakers. Therefore, it needs to become quite large, containing 
many thousands of words for use on text to be analyzed. 

There is another approach which has been used with ver- 
batim transcripts of spoken material and has tended to focus on 
word usage. The computer has been used as a clerical assistant 
to develop a form of specialized dictionary from the material 
itself, rather than from a general-purpose dictionary, many 
words of which will not exist in these documents. This kind of 
approach results in a statistical definition of word groups which 
are associated with each other in an individual speaker’s usage. 
There are studies which have applied this to psychiatric word- 
usage. I mention these approaches to content analysis for the 
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following reason. If one gets into the area of developing simu- 
lated patients or case examples which are conversational in 
nature, one may wish to have tools at hand with which to 
analyze the existing characteristics of language usage for those 
puiposes. Some of these more formal methods of content analysis 
may turn out to be necessary in developing the limited diction- 
ary that you may need to use in the development of curriculum 
materials. 

When we have become involved in the use of computers for 
interaction between man and machine, it becomes necessary to 
find methods for immediate content analysis which can be 
specified and programmed as an entirely automatic process. The 
machine has to have a means of recognizing language responses 
by a subject and of rapidly developing appropriate replies in 
return. We can look upon such activities as attempts to simulate 
human information processes. We can build machine methods 
to imitate human activity in as many respects as possible. Never- 
theless, we use whatever machine methods seem practical at the 
moment, in view of current machine capability and present state 
of knowledge about language. These methods may not look much 
like human information processes. 

There are a number of approaches to this kind of study of 
conversational interaction. I . might remind you of Kenneth Col- 
by’s work; of several other men, Bellman, Friend and Kurland, 
who did some work of this type; and finally of the work at 
MIT of Joseph, Weizenbaum, who has been particularly interested 
in the credibility of a machine response to a subject who is 
interacting in a conversational fashion. In his work we see a 
reliance on immediate context in the development of rules of 
language processing which may apply only to a specific combina- 
tion of attributes — not only of the individual subject being being 
studied but also of the particular point in the conversation. 

My concerns have been similar to his. Conversational ability 
becomes possible if we are able to develop a form of content 
analysis that can deal with language fragments which are often 
incomplete or incorrect from a grammatical standpoint. The kind 
of language processing engaged in by the linguist does not help 
a great deal. Recognition of the meaning of such fragments can 
lead to conversational responses which indicate what we may 
call "understanding” by the computer program. It will be abie 
to make responses that indicate appropriate or reasonable track- 
ing. Responses have to be considered as specific to the context 
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surrounding a particular point in the conversation and they re- 
quire what amount to miniature dictionaries to operate within 

such specific contexts. 

Let me present some further examples. 

Example 7 shows interaction with a simulated dental patient 
that some of the people at the U.C. Medical Center have been 
developing. The format is the same as before, with the periods 
at the left indicating a response by the subject. This program 
acts as though it represents a specific patient coming to a dentist 
and is capable of answering questions corresponding to history- 
taking procedure. 



EXAMPLE 7 

I COMPLAINED TO MY PHYSICIAN ABOUT THIS SORE IN MY 
MOUTH, AND HE REFERRED ME TO YOU. HE WOULD LIKE 
YOUR OPINION. 



. Where is the sore located? 



THE SORE IS ON THE INSIDE OF MY RIGHT CHEEK AND ON 
THE RIGHT SIDE OF MY TONGUE. 

. Have you ever had one like it before? 

NO. 



. How long have you had this one? 

I FIRST NOTICED IT ABOUT 2 MONTHS AGO, AFTER I HAD 
BEEN EATING SOME TACOS. 

. Have you tried any kind of treatment or mouthwash? 

NO, I WAS HOPING IT WOULD GO AWAY. 

. Has it been getting any better? 

IT’S BEEN ABOUT THE SAME EVER SINCE I FIRST NOTICED IT. 
. Does it cause you any trouble eating? 

I CAN’T EAT SPICY FOODS, BUT EVERYTHING ELSE IS OK. 
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. Does anything else make it hurt? 

IT SEEMS WORSE WHEN I EAT SPICY FOODS, AND 
SOMETIMES WHEN I’M NERVOUS. 

. Anything else make it worse? 

I THINK IT’S WORSE WHEN I’M HAVING MY PERIOD. 

. Does smoking bother it? 

I’VE CUT DOWN MY SMOKING BECAUSE THAT SEEMED TO 
IRRITATE MY SORE. 

Example 8 shows the same patient being asked different 
questions. Some of the same information appears. It is clear that 
the student can obtain more information under one strategy than 
another, and therefore can be directed and rewarded, if the clini- 
cal instructor really knows what he wishes to have happen in 
that regard. 



EXAMPLE 8 

I COMPLAINED TO MY PHYSICIAN ABOUT THIS SORE IN MY 
MOUTH, AND HE REFERRED ME TO YOU. HE WOULD LIKE 
YOUR OPINION. 

. Is it a painful sore? 

YES. 

. What kind of pain is it? 

IT’S A BURNING PAIN. 

. Is there anything special that makes it hurt? 

IT SEEMS WORSE WHEN I EAT SPICY FOODS, AND 
SOMETIMES WHEN I’M NERVOUS. 

. Anything else? 

I’M NOT SURE WHAT YOU MEAN, DOCTOR. 
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. Does anything else make it hurt? 

I THINK IT'S WORSE WHEN I’M HAVING MY PERIOD. 

Example 9 provides a third example, in which there seems 
to be a little repetition on the part of the interviewer which de- 
velops a slight resentment in the patient. 

EXAMPLE 9 

I COMPLAINED TO MY PHYSICIAN ABOUT THIS SORE IN MY 
MOUTH, AND HE REFERRED ME TO YOU. HE WOULD LIKE 
YOUR OPINION. 

. How long have you had the sore? 

I FIRST NOTICED IT ABOUT 2 MONTHS AGO, AFTER I HAD 
BEEN EATING SOME TACOS. 

. Is it painful? 

YES. 

. What kind of pain is it? 

IT'S A BURNING PAIN. 

. What kind of a pain? 

IT FEELS LIKE IT IS ON FIRE. 

. Please describe the pain. 

I TOLD YOU, DOCTOR, IT BURNSI 

Now I am going to show you some brief examples of a 
psychiatric patient and, I might say, a somewhat more fully 
developed patient in the sense of having lived longer and inter- 
acted with more interviewers.* 

You will notice that in the case shown in Example 10 there 



♦Starkweather, J.A., Kamp, M., and Monto, A., Psychiatric interview simulation by 
computer. Methods of Information in Medicine . 1967, 6, 15-23. Also in Proc. 7th 
IBM Med, Symposium, 1965, 291-308. 



THE UNIVERSITY COMPUTING CENTER 87 



is a little bit of introductory information which lets the inter- 
viewer know what is expected of him. Once again, our major 
purpose in working with the psychiatric patient in simulating 
this particular kind of case was not only to make an interesting 
demonstration but also to take an area of minimal constraints 
which would push us pretty far in handling a variety of free- 
form responses. Showing just one series of this, of course, does 
not give you a good indication of the variations that can occur 
under the different interviewing strategies. I will just assure 
you, though, that there are a variety of things that can appear 
from this patient. 



EXAMPLE 10 

THE PATIENT ENTERS YOUR OFFICE. HE IS A SLENDER, 
CONVENTIONALLY DRESSED MAN APPEARING ABOUT THIRTY 
YEARS OLD. HIS NAME IS JOHN SANDERS. YOU SEAT 
YOURSELF AND ASK HIM TO BE SEATED. HE SITS DOWN 
AND SAYS . . . 

THANK YOU. 

. How old are you, Mr. Sanders? 

I’M 33. 

. Have you been to a psychiatrist before? 

NO. 

. Why are you here today? 

WELL, MY WIFE WANTED ME TO COME. 

. Your wife? 

IT’S NOT MY IDEA, THAT’S FOR SURE. 

MY WIFE JUST TOLD ME I OUGHT TO SEE A PSYCHIATRIST. 
SHE SAID SHE WAS GOING TO LEAVE IF I DIDN’T GO. 

. Why did she say that? 

I DON’T WANT TO TALK ABOUT MY WIFE. 
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. Why not? 

I JUST DON’T WANT TO TALK ABOUT IT. 



. Well, then tell me about yourself. 

TELL YOU ALL ABOUT MYSELF, HUH? 

. Yes. 

WELL, I’M A PAINTER . . . NOTONE OF THEM ARTISTS, YOU 
KNOW ... A HOUSEPAINTER. MAKE PRETTY GOOD MONEY 
AT IT . . . FOUR SEVENTY-FIVE AN HOUR, AND BEEN 
WORKING PRETTY STEADY, TOO. 

. I see. 

LATELY I’VE BEEN WORKING MOSTLY FOR THIS 
CONTRACTOR, GIANTELLI. WE’RE ON AN APARTMENT JOB 
NOW. TWENTY-TWO UNITS OVER NEAR THE PARK. 

. Please go on. 

SAY, HOW COME YOU WANT TO KNOW SO MUCH ABOUT MY 
JOB FOR? 

. Well, I thought you wanted to tell me about it. 

YOU DON’T THINK THEY’LL FIND OUT AT WORK ABOUT ME 
COMING HERE, DO YOU? 



. Why do you ask? 

IF THEY KNEW I WENT TO SEE A PSYCHIATRIST, THEY’RE 
LIABLE TO THINK THERE’S SOMETHING THE MATTER 
WITH ME. 



. What would they think? 

I DON’T WANT TO GO INTO THAT. 



. All right. 

GEE, DOC, I DON’T KNOW WHAT TO TELL YOU. ASK ME 



■J'Uyacta inoj 
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SOME QUESTIONS, THAT WAY YOU CAN FIND OUT WHAT 

YOU WANT TO KNOW. 

. I need to know more about the reason you are coming to see a 
psychiatrist. 

WELL, I DON’T KNOW . . . ANYWAY, WHAT I CAME ABOUT IS, 

MY WIFE SAYS SHE CAN’T STAND THE WAY I ACT ANYMORE. 

. How do you act? 

WELL, YOU KNOW WOMEN . . . THERE’S ALWAYS SOMETHING 

BUGGING THEM, BUT IT’S PRETTY HARD TO FIGURE OUT 

WHAT IT IS. 

As you can see there is a good deal more. The patient may 
become silent, or difficult in one way or another, and in this 
particular case, he finally walked out. 

In a related way, we have the experience of a medical stu- 
dent who worked this last summer with Dr. Martin Kamp of 
our staff, and developed a simulation of a medical patient who 
complained of shortness of breath. That is, that would be her 
complaint if she were appropriately questioned. I would like to 
quote just a part of the work of this medical student because I 
think you will get some sense of the student’s viewpoint after 
his having had the experience of working on this kind of ma- 
terial. 

It was decided that to have just an interaction between stu- 
dent and patient would not be using the full potential of the 
system. Rather, some means of evaluating and commenting on 
the student’s performance should be incorporated into the pro- 
gram. If the student asks an inappropriate question,. he receives 
a message from the professor. The professor’s comments are 
distinguished from those of the patient by being enclosed in 
asterisks. The program begins as shown in Example 11. 



EXAMPLE 11 

MRS. ERIC KRANKHEIT IS A 34-YEAR-OLD, WHITE, 

MARRIED HOUSEWIFE. YOU ARE THE ATTENDING PHYSICIAN 
AND MUST TAKE THE HISTORY. YOU ARE FORTUNATE FOR 
DR. WATCHIT IS HERE TO MAKE SURE YOU TAKE A GOOD 
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HISTORY. YOU CAN TELL WHEN HE IS SPEAKING FOR HIS 
COMMENTS WILL BE ENCLOSED IN ASTERISKS. 

* HELLO DOCTOR. I’M DR. WATCHIT. GOOD LUCK.* 

YOU MAY BEGIN THE INTERVIEW. 

. What kind of trouble have you been having? 

I’VE BEEN SHORT OF BREATH. 



. When did it start? 

IT STARTED ABOUT FIVE YEARS AGO. 

. What is it like? 

NOW OR WHEN IT FIRST STARTED? 

. When it first started. 

WHEN IT FIRST STARTED IT WASN'T TOO BAD. MY HUSBAND 
AND I LIKE TO HIKE A LOT, AND BEFORE I NEVER HAD ANY 
TROUBLE KEEPING UP WITH HIM. BUT THEN I STARTED 
GETTING SHORT OF BREATH IF WE HAD TO WALK UPHILL A 
LONG DISTANCE, AND I’D HAVE TO STOP AND REST A FEW 
MINUTES. IT KEPT GETTING V/ORSE AND ABOUT THREE 
YEARS AGO I BEGAN TO HAVE TROUBLE CLIMBING 
STAIRS. I’D WALK UP A FLIGHT OR TWO AND BE SO SHORT 
OF BREATH I’D HAVE TO STOP AND REST AWHILE. 



The student who wrote this program said that he recognized 
that each physician or medical student has his own individual 
method of history-taking and that although these methods differ 
in sequence and wording, they all might get the same amount 
of information. He wanted to limit the complexity of the pro- 
gram— and get through with his work during the summer— so 
he decided to put certain restrictions on the sequence of ques- 
tions that could be asked. 

He permitted the interviewer to ask about various symp- 
toms, but also required that a number of basic questions be 
asked about one symptom before asking about another symptom. 
That is, he restricted the system by requiring that one area had 
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to be explored before going on to another. The number of basic 
questions varied from four to eight depending upon the symptom. 

On the other hand, there were some symptoms which he 
felt were so closely allied that he allowed them to interrelate. 
Here the interviewer could, in fact, jump around in an area of 
two or three related symptoms, such as cough, sputum, and 
blood. 

Example 12 gives an idea of how Dr. Watchit came into 
the picture now and then. 

EXAMPLE 12 

. Do you have a heart murmur? 

WHAT? 

. Have you ever been told that you have a heart murmur? 

*THERE ARE STILL SOME QUESTIONS YOU SHOULD ASK 
ABOUT HER HEMOPTYSIS* 

. What are they, Dr. Watchit? 

* WHY DON'T YOU ASK IF SHE NOTICED ANYTHING THAT 
WOULD PRECIPITATE OR BRING ON THE ATTACKS?* 

. Did you notice anything that would precipitate the attacks? 

I DON'T UNDERSTAND. 

. Did you notice anything that would bring on the attacks? 

NOTHING. THEY’D JUST START AND STOP BY THEMSELVES. 

Once again we have some kind of mechanism for the hand- 
ling of unrecognized questions and it seems to work in this 
instance. 

The consultant (Dr. Watchit) later on advises on other 
possibilities as well, as when the interviewer asks, "Do you have 
any trouble with your vision?” 

Answer: "What?” 

Question: "Do your eyes bother you?” 

Here Dr, Watchit says, "Doctor, *are you considering the 
possibility that the patient may have some kind of heart trouble 
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such as failure, valvular disease, or maybe even ischemic heart 
disease? ” 

The student answers, " Should I be?” 

'"Would you please answer my question with either yes or 

no.” 

Answer: "No,” 

"I think you should entertain the possibility. It would be 
helpful to question the patient further and explore this possibil- 
ity.” 

We have already seen an example in the dental patient of 
what may happen as a result of repetitive kinds of questioning. 
Here is an example, where the student asks, "Do you ever cough 
up any blood?” 

Answer: "Yes.” 

Question: "Is it bright red or sort of brownish?” 

Answer: "I don’t remember too well but I think it was 
bright red.” 

Question: "What color was it?” 

Answer: "Like I said, it was bright red— I think.” 

Question: "Was it bright red?” 

Answer: "How many times are you going to ask me that?” 
Question: "What color did you say it was?” 

Dr. Watchit then comments, "You had best go on to some- 
thing else.” 

You can see that the student-author had fun and learned a 
good deal from this situation. Now let me give you a brief ex- 
ample of how some of this is done. 



EXAMPLE 13 

PILOT CODING 

T: PLEASE TYPE YOUR FIRST NAME.; 
LABEL: A: ; 

T: GLAD TO MEET YOU, “LABEL".; 
PILOT CONVERSATION 
PLEASE TYPE YOUR FIRST NAME. 
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. Fred 

GLAD TO MEET YOU, FRED. 

At the top of Example 13, there is some coding in PILOT 
which indicates, "I want computer to type”— that is what the 
"T” means. Then the program asks, "Please type your first 
name” and it sets up the answer. The program is going to ac- 
cept future answers and store them by name, which in this 
case is "Label,” but could be anything. Then it proceeds to 
type, "Glad to meet you, 'Label.’” This means that it will 
retrieve whatever name was stored. 

The conversation resulting from this asks the student, 
"Please type your first name.” He types in "Fred.” It then 
says, "Glad to meet you, Fred.” This is an example of how all 
or a portion of a person’s response can be stored away under 
a name and brought back at a later point in the program. The 
program can get very friendly by this means. 

Example 14 shows three different interactions with a portion 
of a question-type program. You can see this material handled 
in three different ways. The first case is a correct answer; the 
second case is a recognizable but not completely correct response, 
which gets some help; and the third case is an unrecognized 
answer. 

EXAMPLE 14 

CONVERSATION WITH A PILOT PROGRAM 

(1) A FEMUR IS A . 

(TYPE A WORD) 

. bone 

THAT IS CORRECT. 

(2) A FEMUR IS A . 

(TYPE A WORD) 

. part of the thigh 

YOU ARE CLOSE, TRY AGAIN. 

. one of the leg bones 
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THAT IS CORRECT. 

(3) A FEMUR IS A . 

(TYPE A WORD) 

. small furry animal 

SORRY, TRY AGAIN. 



. rodent 



SORRY, TRY AGAIN. 

Example 15 demonstrates the kind of coding that is capable 
of producing those three different forms of output. The computer 
types the first two lines, and then accepts an answer. In this 
instance, the recognition item is "bone” and if "bone’ a PP® ars 
there is a positive response, i.e., recognition, which says, "That 
is correct.” Having gotten to that point, there is a control item 
which says, "GJUMP TO END.” This means that if there is 
recognition, that is if the good response appears, it is going to 
jump to "End” for control of the program. 

EXAMPLE 15 

AN EXAMPLE OF PILOT CODING 

T: A FEMUR IS A ---- . ; 

T: (TYPE A WORD) ; 

GO: A: ; 

R: BONE; 

G: THAT IS CORRECT.; 

C: GJUMP TO END; 

R: LEG, THIGH, HIP; 

G: YOU ARE CLOSE, TRY AGAIN.; 
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C: JUMP TOGO; 



END: E: ; 

In the program shown in Example 16, items are recognized 
lor the first and second responses; but for the third response, 
the word "uncommon” is given as the answer. It is not recog- 
nized and the computer types, "Try it again. What kind of 
characteristics differentiate people from one another?” 



EXAMPLE 16 

THE REASON WE ARE ABLE TO FORMULATE SOME GENERAL 
PRINCIPLES ABOUT HUMAN NATURE IS THAT THE BASIC 

HUMAN MOTIVATIONS ARE BY ALL 

PERSONS. 

. known 

THAT SOUNDS LIKE YOU MAY HAVE THE RIGHT IDEA, BUT 
I CAN’T BE SURE. PLEASE TRY AGAIN FOR A MORE 
SPECIFIC WORD. 

. shared 

RIGHT, AND AN EXAMPLE OF THESE "SHARED” 
MOTIVATIONS IS THE UNIVERSAL DESIRE FOR SATISFYING 
INVOLVEMENT WITH OTHER HUMAN BEINGS. 

IN ADDITION TO THE COMMON FEATURES, EACH PERSON 

ALSO HAS HIS CHARACTERISTICS WHICH 

DIFFERENTIATE HIM FROM EVERY OTHER PERSON. 




. uncommon 

TRY IT AGAIN. WHAT KIND OF CHARACTERISTICS 
DIFFERENTIATE PEOPLE FROM ONE ANOTHER? 

personal 

YES, THESE ARE PART OF THE PERSONALITY, BUT THE IDEA 
IS THAT EACH PERSON HAS "UNIQUE” 

CHARACTERISTICS. 
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The author does something like the following in reviewing 
this operation. (See Example 17.) He enters an edit mode and 
the computer indicates that it is in an edit mode. The author 
then asks for a display of the non-matching items, where it 
says, "display nomatch.” It displays question two and the 
answer, "uncommon.” Moving on, he types, "Display the 'R 
items for number two,” and it displays a series of different 
word groups labeled R2A, B, C, D, and E. These uifferent 
groups would, in the program itself, undoubtedly produce differ- 
ent kinds of output depending upon which group contained the 
word that was recognized in the response. The author here 
wants to add the word "uncommon” to one of these lists. He 
says, in this edit mode, "I want to add to R item 2A, the 
word 'uncommon.’ ” He also adds it to 'R item 2E. 

He then asks, "Display once again the *R’ items for ques- 
tion 2,” and here you see that "uncommon” has been added 
at level A and level E. The author goes back and runs the 
program again, entering it at question 2. The computer resumes 
execution of the program and puts out the lead-in for that ques- 
tion. The response "uncommon” is given and this time it rec- 
ognizes it and says, "Yes, people usually differ. 



EXAMPLE 17 



. aaaedit 

^ EDIT MODE*** 

. display nomatch 

Q2 — UNCOMMON 
***EDIT MODE*** 



. display r 2 

R2A- UNIQUE, DIFFERENT, INDIVIDUAL* 
R2B — PERSONAL* 

R2C — SEPARATE 
R2D — OWN 

R2E - DIFFERENT, UNIQUE, INDIVIDUAL* 
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***EDIT MODE*** 

. add i 2 a 

ENTER ADDITIONAL R-ITEM(S) 
. , uncommon 

***EDIT MODE*** 

. add r 2 e 

ENTER ADDITIONAL R-ITEM(S) 
. , uncommon 

***EDIT MODE*** 



. disp.’c.y r 2 

R2A — UNIQUE, DIFFERENT, INDIVIDUAL*, UNCOMMON 
R2B - PERSONAL* 

R2C - SEPARATE 
R2D — OWN 

R2E — DIFFERENT, UNIQUE, INDIVIDUAL*, UNCOMMON 
***EDIT MODE*** 



. endedit aaa2 l * 

IN ADDITION TO THE COMMON FEATURES, EACH PERSON 

ALSO HAS HIS CHARACTERISTICS WHICH 

DIFFERENTIATE HIM FROM EVERY OTHER PERSON. 



. uncommon 

YES, PEOPLE USUALLY DIFFER FROM EACH OTHER IN 
MANY RESPECTS. 



This is an indication of the kind of mechanism being de- 
veloped to facilitate the improvement of programs— a process 
that recurs again and again in the development of these instru- 
ments. Authors must have a relatively easy method for doing 
this. 
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We would like to be in a position to explore a variety of 
terminal devices and the possible role of PILOT as a communi- 
cations link to many kinds of programs, not just as a CAI 
language for instruction in a direct fashion. We would like to 
be able to question a person who comes on a terminal and does 
not know how to use the machine, assess his problem and act 
as a consultant to help him get his work done. 

There are many developments in control systems for inter- 
active computers. Most require specialized knowledge in order to 
learn the codes necessary to gain entry to the systems and make 
good use of them. We hope to use our free response, interactive, 
conversational language to get around this problem. For the user, 
who might be a student or a member of the faculty, the com- 
puter will then be an aid to learning over which he will have 
a considerable amount of control. 
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COMPUTERS IN DENTAL SCHOOL ADMISSIONS 



Thomas J. Gmley 
Council on Dental Education 
American Dental Association 

u r! n 19 . 75 : , a PP r ? ximatel y 15,000 applicants will participate in 
innnnn ta 4 . ‘. ss * on Testing Program and will submit at least 
100,000 applications to dental schools. In 1969-70, the 11 000 
candidates submitted approximately 60,000 applications to’ the 
dental schools in the United States. There is no question that, 
as the number of candidates continues to increase, each will find 
acceptance by his preferred school more and more difficult to 
obtain. As a direct result, the dental applicants are forced to 
apply to an increasing number of schools in order to ensure 

consideration and eventual admission to some dental education 
program. 

From an administrative and clerical standpoint, the paper- 
work demanded in processing this volume of candidates exceeds 
the normal capabilities of admission committees. The magnitude 
of the problem becomes apparent when one realizes that these 
candidates will submit not only 100,000 aptitude test scores but 
in all likelihood will submit 100,000 transcripts from undergrad- 
uate colleges, 200,000 letters of recommendation, and a variety 
of other documents and statements pertaining to their admission 
qualifications. 

The sheer size of these numbers will confront the admission 
committees of the various dental schools not only with a magni- 
fied clerical problem but, more importantly, with more difficult 
decisions in the selection of candidates for the dental education 
programs. Therefore, if the dental schools in the future are 
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going to continue to select the most qualified candidates, a sys- j 

tern must be developed to meet the expanding admissions prob- '] 

lem by providing dental schools with a procedure superior to the j 

method currently in use. | 

Potential solutions to the problem now exist, if one is will- j 
ing to consider the alternatives available. It might be possible, 
for example, to establish a system similar to the National Intern- ^ 

ship Matching Program. This system allows medical school 
graduates and hospital administrators to match their intern pref- 
erences on a national basis. The candidates list, in the order of • 
preference, the hospital internship appointments of their choice ! 
and the subscribing hospitals have an opportunity to also indi- 
cate the candidates of their choice. The matching procedures are 
accomplished through the use of computers. Although there were [ 

many initial problems related to the development of the National .] 

Internship Matching Program, it is generally agreed that, on the 
whole, the hospitals are pleased with the results. j 

It is conceivable that dental schools could cooperate in a j 
similar venture. It would be possible, as has been suggested in ] 
numerous papers presented at annual meetings of the American 
Association of Dental Schools, to develop a centralized application : 

system for dental school applicants. Such a system could be i 

installed in a progressive manner as it earned acceptance by the j 

dental schools. For example, the dental schools would not ini- 
tially use a computer admission program to select their students. 

The system’s first phase would be a clearinghouse whereby can- 
didates apply once to a central system which would then process f 

all of the records and forward duplicate copies to the institutions. j 

In this way at least a portion of the clerical work would be 
diminished on the part of both the candidates and the individual i 
dental schools. This initial system, however, would be least ef- 
fective in reducing the total clerical work of the dental schools j 

but it would reduce the workload of the undergraduate colleges 'j 

and applicants in providing transcripts and letters of recommen- ■; 

dation . j 

A second level of an application system could involve initial j 

screening of candidates. In this instance, not only would the j 

centralized applicant system clerically process the papers and f 

reports necessary for each of the schools, but would also screen j 

out those individuals who do not meet the established pre-admis- j 

sion requirements of the individual institution. For example, j 

applicants might apply to Emory University’s School of Dentistry j 
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through the application system. If Emory was then requiring 
three years of pre-professional education prior to considering any 
candidate for admission, all individuals applying to that institu- 
tion with less than three years of pre-professional education 
would be eliminated. As another example, a state dental school 
might accept students only from within the state. Nonresident 
applicants, therefore, could be screened out automatically through 
the use of this computerized application system. Through the use 
of this second level system, dental schools would received a re- 
duced number of applications to process and consider for admis- 
sion, but final selection would be left exclusively to the indivi- 
dual institution. 

The third level, and the least acceptable to the dental 
schools at the present time, would be the actual selection of 
candidates by the use of a computer. While there is no question 
that computer selection is possible, I. am sure we would all agree 
that the final decision governing acceptance to a professional 
school should not be based solely on formulas included in a com- 
puter program. There is, however, an increasing awareness that 
something must be done in the area of admissions, and perhaps 
a compromise system can be developed with the cooperation of 
all of the dental schools. 

The Association of American Medical Colleges recently in- 
troduced, on a pilot basis, a centralized application system. Sev- 
eral medical schools cooperate by accepting application informa- 
tion from a central office rather than compiling it on an 
individual basis. While the results of this pilot venture will un- 
doubtedly be interesting and applicable to the dental education 
process, it is unacceptable in my view to try to introduce such 
a system by the means of pilot programs. If the dental schools 
are interested in developing, on a national basis, a centralized 
system for processing dental school applications, it is essential 
that all of the dental schools cooperate in the initial venture. I 
would suggest that in the beginning, only the first two phases of 
the program outlined in this paper be considered by the Ameri- 
can Association of Dental Schools. The proposed system would 
not make final selection of candidates to dental school but would 
only screen out individuals unacceptable to each institution. 
Secondly, the dental schools would receive a computer print-out 
of all applicants. The listing i would include traditional informa- 
tion such as undergraduate course and grade point average, (pre- 
professional) institutional identification, dental aptitude 
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test scores, letters of recommendation, and other miscellaneous 
information viewed as essential by the dental schools. 

As an initial step, however, it would be necessary for all 
the dental schools to accept a common application form. Also, it 
would be necessary that the schools not require original trans* 
scripts from the undergraduate colleges, but accept the computer 
print-out from the national application system. This might create 
certain legal and administrative difficulties in universities, but 
probably these problems could be solved in such a way that a 
national program would be acceptable to all dental schools. 
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OHIO STATE REGIONAL MEDICAL PROGRAM 
COMPUTER-ASSISTED INSTRUCTION STUDY 



Daniel E. Tira 
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Computer-assisted instruction (CAI) at the Ohio State Uni- 
versity is at present supported through the combined efforts of 
the College of Dentistry, the College of Medicine, and the Ohio 
State Regional Medical Program. As it originally developed on 
this campus, CAI was solely the undertaking of the College of 
Medicine. Later, the College of Dentistry and the Regional Medi- 
cal Program became involved. 

Early in 1965 the medical-dietetic section of the College of 
Medicine developed a series of computer-based exercises to assist 
hospital dietitians in determining nutritive values for special diet 
planning. Although not as instructional in nature as a didactic 
program developed for use via a computer system would be, these 
exercises generated an appreciation for and an interest in the 
total potential value of CAI as a viable instructional method at 
Ohio State in the health areas. 

An outgrowth of this experience was the design and develop- 
ment of a CAI program of review material in gross anatomy for 
medical students on an independent study track within the Col- 
lege of Medicine. This CAI material was identified and written 
by medical students who had just completed the course in gross 
anatomy. It was well received as "tutorial self-evaluation” by 
the students. It answered (on an individualized basis) their need 
to know their progress in relation to their electing to remain in 
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the traditional lecture/dissecting section track. This course has 
been rewritten recently and is now us$ed in conjunction with the 
anatomy program in the University Medical School. 

In 1968, the Department of Biochemistry gave released time 
to a faculty member to write tutorial self-evaluation CAI material 
for dental students in a course on physiological chemistry. Dr. 
Helen Wikoff completed this material during the fall quarter of 
1968 for use by the entire sophomore class in the College of 
Dentistry. The experience was informative. The sophomore class 
numbers 155 students at Ohio State but only three 1050 IBM 
terminals were available to the students. 

Scheduling for all students was nearly impossible, despite 
an 8:00 a.m. to 10:00 p.m., seven-day s-a- week, CAI operating 
schedule, and was complicated by the fact that 60 medical stu- 
dents were also using the system and the same terminals for 
gross anatomy, and by the fact that the 8:00 a.m. to 5:00 p.m. 
use was sporadic due to other scheduled classes. Needless to say, 
it became chaotic at times, since both CAI courses are approxi- 
mately 14 hours long. 

Student response and attitudes varied. The course material 
was well written and slanted toward practical, clinical, dental 
application. However, some students felt they needed to beat the 
system, and did not fully realize that the material was an aid 
in self-evaluation of their own progress in the course. Consider- 
ing that self-evaluation is a new concept to students who are 
used to a lock-step curriculum, especially in dentistry, the initial 
attitudes were not that surprising. Many students did feel it was 
an aid in understanding the material and an excellent review 
medium prior to examinations. 

In August of 1968, the Ohio State Regional Medical Pro- 
gram was funded for a feasibility study to test the use and use- 
fulness of CAI in meeting the continuing education information 
needs of health professionals in the local community. The Uni- 
versity Hospital’s IBM 360/40 disk operating system (used for 
the courses just described) was the basis for a network connect- 
ing four community hospitals in central Ohio by data telephone 
line. IBM 1050 teletypewriter terminals were placed in each hos- 
pital. This network presently includes a total of ten hospitals 
throughout Ohio. Course materials developed for this project in 
its initial year included Care of the Patient with Coronary Heart 
Disease , Anticoagulant Medication , Stroke Rehabilitation , and 
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Diagnosing the Red Eye. These and the Medical College programs 
became available for use at all terminals. 

A course entitled Oral Cancer Recognition was planned and 
written by Drs. Robert Finch, C.J. Cavalaris, and Gerald Gas- 
ton from the Divisions of Oral Pathology and Oral Surgery at 
the Ohio State University College of Dentistry, early last spring. 
Since this program is directed quite aptly to a group such as is 
gathered here, I will expand on some of the notions on which 
this course was developed and then give a very brief illustration 
of one aspect of the total course sequence. 

The course authors designed the Oral Cancer Recognition 
course to reflect, as nearly as possible, the clinical practice of 
dentistry and medicine involved in diagnosis making. A simula- 
tion sequence was established for six clinical cases to permit the 
learner to interact, as in a clinico-pathologic conference, with the 
computer program to develop a freely structured history, physical 
examination, and laboratory data. When sufficient information 
upon which to base a diagnosis is obtained by the learner, he 
makes his choice of diagnosis and the instructional program gives 
appropriate feedback. In the event of an incorrect choice, feed- 
back directs the learner’s thinking regarding subsequent choices 
towards the correct diagnosis. Two cases require differentiating 
three diagnoses in order of suspicion, and four cases require 
only the primary diagnosis. 

A pretest portion consists of two cases wherein the learner 
develops the patient history, physical examination, and labora- 
tory findings by questioning the computer program, as if it were 
the patient or the laboratory. The choice of differential diagnosis 
is stored in the computer’s memory. Later, in the post-test, the 
learner may alter this differential diagnosis after completing the 
intervening course of instruction. The first part of the instruc- 
tional material reviews normal findings, a systematic approach 
toward oral examination, and the appearance of benign and 
malignant oral lesions. The second part of the course of instruc- 
tion consists of four cases using lesions which have teaching 
value regarding salient points in oral cancer detection. The learn- 
er has the option to take additional material in three subjects: 

1. Oral Biopsy and Referral for Cancer Treatment. 

2. Oral Exfoliative Cytology. 

3. Statistics Regarding Oral Cancer. 

The course requires two and one-half to four and one-half hours 
to complete. 
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This simulation of office practice is based on a 125-item 
listing of the parts of a history, physical exam, and laboratory 
tests to which code numbers have been assigned. In a similar 
fashion, disease entities and categories were number coded. These 
came to a total of 115 entries. The user selects his requests for 
information in any order he deems necessary. When the diagno- 
sis choice is made, further information cannot be obtained from 
the history, physical or laboratory data. 

The course authors realized that use of a listing would struc- 
ture the responses by the user and that a free-response mode 
would have been more ideal, especially in a diagnostic situation. 
In the near future, it is possible that this program will be re- 
vised to accommodate free response; however, as an initial at- 
tempt at CAI course writing the authors felt that use of a listing 
permitted simpler programming than if freely structured requests 
for information were used, and still satisfied the course intent. 

As an illustration, I will simulate one of the case studies 
that is given in the course. I will give the information that the 
user receives upon request. 

To begin a case study, the user types 099. (A booklet listing 
the possible areas of request is located next to each terminal.) He 
gets a feedback from the computer which says, "Chief Complaint: 

Look at Figure .” The comment is, "I keep biting this 

sore on my tongue.” 

The user then looks at the book of listings. He chooses 
100, which asks what the present illness is. The computer re- 
sponds with, "Patient states that the lesion area was bitten two 
months ago while eating. She says that she continually irritates 
it and it is getting larger.” 

The user then asks if it is painful by typing 103. The com- 
puter responds with, "Painful at times, especially when bitten.” 
The user then wishes to ask, "Has there been a weight change?” 
He types in 105. The computer responds, "None.” 

The user then asks for a physical examination, specifically 
of the tongue, and types in the appropriate number, 418. The 
computer responds with, "Left lateral border of tongue, soft, 
movable, exophytic mass, 2 x 1.5 x 1 cm., easily bleeding on 
palpation.” 

The user now would like information regarding the lymph 
nodes, and types 427. His reply from the computer: "Left sub- 
mandibular lumph nodes small, movable, and non-tender.” The 
user then asks for a general history and types 200. The compu- 
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ter responds with, "This 29-year-old Negro patient states that 
she is enjoying good health, and denies any severe disease or 
injury in the past. She has had two uneventful pregnancies and 
deliveries. T and A at age 4, broken arm at age 3. Hay fever 
in summer. No sensitivities to medicine. Has chronic sinusitis." 

Now the user asks for an exfoliative cytology, and types the 
corresponding number. The computer responds with, "Yes, but 
the specimen will be quite bloody, and cytological examination 
will be less than optimal." 

The user, in that case, asks for a surgical incisional biopsy, 
which is 731. The computer responds again, "No, the lesion is 
small enough that it should be excised with a good margin of 
normal tissue. A pathology report will not be given. Base your 
diagnoses on history and appearance." 

At this point, the user feels he has enough information and 
he types in 999, signifying he is ready to make a diagnosis. The 
user looks at the list, decides the proper diagnosis is squamous 
cell carcinoma, and types the corresponding number, 4121. The 
computer responds with, "You chose: Squamous cell carcinoma. 
Your index of suspicion for cancer is to be commended, but sev- 
eral factors, admittedly tenuous, also suggest other possibilities. 
In differentiating this from tumor we may use the following 
parameters: 

1. Its texture: soft. 

2. Its lack of deep fixation to the tongue (that is, its mobil- 
ity). 

3. An explicable origin in trauma, and continued progres- 
sion by the same mechanism. 

A biopsy is indicated, however. Select another category." 

The user then feels that it might be focal acute inflammation 
and types 1120. The computer responds with, "You chose: Focal 
acute inflammation. Yes, now which one is it? Try again." 

The user now becomes more specific and chooses pyogenic 
granuloma. The computer responds with, "You chose: Pyogenic 
granuloma. Yes, of course. This was a difficult diagnosis because 
this lesion is of unknown etiology, but is thought to be of low- 
grade infectious and traumatic origin. The histologic picture is 
that of endothelial hyperplasia. Its differentiation from tumor is 
difficult due to its exuberant growth characteristics. Therefore, it 
must be removed for pathologic examination, and, because tumor 
is usually in the differential diagnosis, it must be removed with- 
out delay." 
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Feedback-response to the selected diagnosis will give the rea- 
sons for or against the correctness of choice together with a clue 
to the proper diagnosis. It is this portion of the course which is 
most popular with the users. 

It should be noted that this course was directed primarily 
to the practicing dentist or physician. If the course material 
were primarily designed for student use, rather than for the con- 
tinuing education of health practitioners, it would have been a 
relatively simple matter to program the information-gathering sec- 
tion (the history, physical and laboratory data) to store the re- 
quests from the learner. Then, at the time of diagnosis selection, 
the program would indicate that insufficient information had been 
obtained, or that a different sequence of information gathering 
might have been more efficient. 1 

This form of presentation is believed to be perhaps the best 
available method for training future practitioners in the art of 
history-taking, short of a one-to-one, teacher-student team for pa- 
tient evaluation. Working in a team, however, still does not 
guarantee every student adequate exposure to complicated or 
unusual cases. The computer memory is an every-handy reposi- 
tory for this type of material. 

Now *hat I have given you a rough sketch of the history of 
CAI at Ohio State, I would like to give you some notions of 
what we, at least tentatively, envision as the role of CAI and 
other forms of educational technology in the Ohio State Univer- 
sity College of Dentistry. 

At present, the College of Dentistry is undertaking a major 
curriculum change, planned for implementation in 1972, to coin- 
cide with an increase in enrollment from 155 to 205 entering 
freshmen. A faculty education program to acquaint instructors 
with the adjunct utilization of CAI and other forms of educational 
technology has been planned. This program will take the form 
of two series of faculty seminars. One will discuss the various 
categories of educational technology that are realistically applic- 
able in dentistry. Major emphasis, however, will be placed on 
CAI. CAI programs now available to dental students will be re- 
viewed from the standpoint of course development. 

The second series of seminars will involve those faculty 
members who have demonstrated an interest in utilizing CAI. 
Included in these sessions will be discussions of program strategy 
or strategies relative to specific course objectives, and of program- 
ming languages (especially COURSEWRITER III, Version II). 



117 



OHIO STATE CAI STUDY 109 



Furthermore, particular assistance in program preparation will 
be given to each interested faculty member* Five courses will be 
written by five departments within the College of Dentistry as 
demonstration programs. 

As a result of this experience, a decision will be made by 
the faculty whether or not to develop in the Dental College a 
learning laboratory based on CAL This laboratory will develop 
and function in coordination with the revised dental curriculum. 

If accepted, CAJ. will become a tool for student guidance, 
counseling, and tutoring during his dental training. Moreover, 
simulation programming for unusual or complicated patient-man- 
agement situations should provide a much needed variety of 
experiences for dental students. Such simulations will provide 
valuable assistance in acquiring the skills of history-taking, diag- 
nostic decision-making and management decisions. 

The problems of increased enrollment, of a higher student- 
faculty ratio, and the lessened chance for each student to receive 
a full measure of clinical experience, certainly warrant the de- 
velopment of simulated clinical experiences. They also warrant 
the use of CAI as a means for students’ self-evaluation of their 
studies, for drill and practice, and as a source of evaluation 
information upon which to base course content, student counsel- 
ing and tutoring by faculty members. 



118 



CLIFTON E. CRANDELL ia Director of (fie Dental Data Center and Associate Pro- 
fessor of Oral Diagnosis at the University of North Carolina School of Dentistry. He 
holds a B.S. from East Carolina College, a D.D.S. from the Medical College of 
Virginia, an M.S. in Oral Roentgenology from the University of Pennsylvania, and 
an M.Ed. from Duke University. For several years he has been involved in intro- 
ducing the concept of comprehensive patient care into clinical dental education and, 
using linear programming techniques, recently developed a scheduling model for the 
implementation of comprehensive patient care within the undergraduate clinical pro- 
gram at North Carolina. He directed the Conference on Comprehensive Patient Care 
in Clinical Dental Education held at Pine Needles, North Carolina, in 1969. He has 
maintained an active interest in his professional specialty, including serving as Presi- 
dent of the American Academy of Dental Radiology and as Chairman of the First 
International Research Conference in Oral Roentgenology. 



LINEAR PROGRAMMING AND 
COMPREHENSIVE PATIENT CARE 
IN CLINICAL DENTAL EDUCATION 
AT THE UNIVERSITY OF NORTH CAROLINA 



o 

ER AC 



Clifton E. Crandell 
University of North Carolina 

Linear programming is a mathematical technique which can 
be used for determining optimum allocation of resources, such as 
capital, raw materials, manpower, plant, or other facilities. An 
objective might be to minimize cost; it might be to maximize 
time in a teaching situation. Of course, there have to be alter- 
nate choices that can be made or else there is no linear pro- 
gramming problem. 

Linear programming is only a small part of the field of 
operations research, which is, by and large, a computer-based 
set of techniques that have been used extensively by the Depart- 
ment of Defense and by such industries as the petroleum industry 
to find solutions to problems that have a very large number of 
variables. These techniques have seldom been used in education 
and are just now becoming of interest in health care delivery 
systems. Until World War II these techniques remained relatively 
undeveloped. Computer technology has made it possible to develop 
this field to its present level of sophistication. 

The objectives of this project at the University of North 
Carolina have been three- fold. In the first place, we wanted to 
demonstrate the merits of linear programming for the optimum 
allocation, scheduling, and utilization of teaching staff and physi- 
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cal facilities in clinical dental education. Secondly, we wanted to 
demonstrate a practical application— remember we said that you 
had to have some variables and choices you could make— by 
introducing the total patient care concept and studying what effect 
this had on the model. Thirdly, we wanted to make these data 
available to other schools, and that is why I am here today. 

The introduction of the concept of total patient care will pro- 
vide students, we hope, with a clinical experience in dental 
school which will closely simulate the conditions to be encounter- 
ed in private practice after graduation. Linear programming 
techniques will permit us to realize a system of clinical dental 
education in which students provide comprehensive care to pa- 
tients, as opposed to the traditional point and block systems of 
clinical teaching which provide fragmented care. 

Specifically, our pilot program at the University of North 
Carolina has sought to demonstrate that this concept can be in- 
troduced without lowering the quality of instruction or increasing 
the cost of clinical teaching. We hope to identify some parame- 
ters that will enable us to improve dental teaching and actually 
spend less money. Both of these are worthy objectives that will 
benefit both the student and the faculty, as well as the popula- 
tion that we serve. 

In carrying out this project over the past three years, work 
has proceeded along several lines. First, we analyzed the clinical 
teaching situation, using a systems approach, and identified the 
factors that are significantly related to the effectiveness and effi- 
ciency of clinical instruction. Included was an analysis of tech- 
nique procedures that students are required to accomplish, the 
classification of students in terms of ability, the time required 
with reference to these procedures, the rate of utilization of equip- 
ment and physical facilities, the rate of utilization of instructors’ 
services, and a classification of patients in terms of the complex- 
ity of their dental needs. 

Secondly, we assumed that the relationships between these 
factors were linear. I should add, at this point, that there are 
also nonlinear programming techniques. Nevertheless, we assum- 
ed for our case that the relationships are linear. We simulated 
these factors conceptually in a mathematical model, and the in- 
formation was fed into the computer as representing the clinical 
situation. 

We have identified variables which we feel might represent 
the dental clinic. The problem for the computer was to determine 
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which set of conditions would give us the most efficient operation 
of the dental clinic. Once we are satisfied that our mathematical 
model best represents the clinical situation, and this is not an 
easy point to reach, we will then experiment with changes in 
the operation of the clinic within the computer without making 
any changes in the real-world or clinical situation. This is one 
of the chief advantages of the technique. Once you get a satisfac- 
tory model, the cost of experimentation is very little, certainly 
much less than building a new building or hiring an entirely 
new faculty. 

LINEAR PROGRAMMING: FINDING THE OPTIMUM 
WITHIN THE FEASIBLE 

With thanks to IBM, I would like to cite a very simple 
example of linear programming and how it can be used. 

Let us consider, in this example, a small machine shop 
which manufactures a standard model and a deluxe model of an 
unspecified product. Each standard model of the product requires 
four hours of grinding and two hours of polishing, while each 
deluxe model requires two hours of grinding and five hours of 
polishing. The manufacturer has two grinders and three polish- 
ers. Therefore, in the 40-hour week, he has 80 hours of grind- 
ing capacity and 120 hours of polishing capacity. He makes a 
profit of $3.00 on each standard model and $4.00 on each de- 
luxe model. 

The question might be, how should the manufacturer allo- 
cate his production capacity to achieve the most profit? The first 
step is to convert this problem statement into mathematical form. 
We let the symbol "S” stand fpr the number of standard models 
manufactured in a week, a convenient time period to work in, 
and let re D” represent the weekly number of deluxe models man- 
ufactured. By their definitions, "S’* and "D” are greater than 
or equal to zero. The profit for making "S” standard models 
and "D” deluxe models in a week can be expressed as 

profit = (3S + 4D) dollars. 

Now if five standard models, S=5, and seven deluxe models, 
D=7, are built in a week, 

profit = 3x5 -f 4x7 = $43.00. 

If the manufacturer could make 25 standard models and 20 de- 
luxe models, the profit would be $155.00. 
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How can we express the restrictions on machine capacity? 
The manufacturer of each standard model uses four hours of 
grinding time. Making "S” standard models therefore requires 
4S hours. Similarly, the manufacturer of "D” deluxe models 
uses 2D hours since the manufacture of one deluxe model uses 
two hours of grinding time. The total number of hours of grind- 
ing capacity used in a week is 4S plus 2D. Since we have al- 
ready said that 80 hours of grinding time were available, we 
might be tempted to write 

4S 4- 2D = 80 hours. 

This formula would be incorrect because we have no assurance 
at this point that the greatest profit will result by using all the 
grinding time. All we know is that the total grinding time must 
not exceed 80. Conveniently, this can be expressed as 
4S 4- 2D ^ 80 hours. 

In the same manner, we arrive at the limitation of polisher 
capacity 



2S 4- 5D ^ 120 hours. 

Since a higher profit per unit can be made on deluxe models, 
one might expect that the optimum policy would be to make as 
many deluxe models as possible and to forget about the standard 
models. . This may not be correct. Let us calculate how many 
deluxe models alone could be made and record corresponding 
profits for future reference. The limitation on grinder time pro- 
vides that two times the number of deluxe models must not ex- 
ceed 80 and the limitation on polisher time provides that five 
times the number, of deluxe models must not exceed 120. The 
grinder capacity permits 40 deluxe models to be made; polisher 
capacity permits no more than 24 to be made. Therefore, 24 is 
the maximum number of deluxe models that could be made even 
though this policy would only consume 48 hours of the grinder 
time out of the 80 that are available. The profit with this policy 
then would be $96.00, since there is a $4.00 profit on each of 
the 24 deluxe models. 

Similarly, if the policy of producing just standard models is 
chosen, only 20 models could be produced in a week because of 
the limitation imposed by grinder capacity. With this policy, 
profits would be $60.00 and only 40 of the 120 hours of polish- 
er capacity would be used. Although the first policy results in a 
higher profit, these are not the only two alternatives. 

Many other policies might be chosen in which some mixture 
of standard and deluxe models were produced. For each mixture 
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of model types proposed, it would have to be determined whether 
grinder and polisher capacities were sufficient and, if so, what 
the profit would be. Eventually, the policy or policies returning 
the greatest profit would be found by considering, in turn, each 
possible policy. Linear programming provides the framework for 
solving the manufacturer’s problem of choosing a policy that will 
maximize profits, but the computer assists greatly by assuming 
the burden of calculation. 

Conceptually, the solution process can be appreciated more 
easily with the aid of a graph. Figure 1 shows an area bounded 
by two axes. The vertical axis represents S, or the number 
of standard models produced. The horizontal axis represents 
”D,” or deluxe models produced. Any point placed on the 
graph would represent one week’s production of the two types of 
models. Figure 1 also shows the constraint imposed by grinder 
capacity, which has already been expressed as 
4S + 2D ^ 80 hours. 

Any point below the line represents a specific combination 
of "S” and "D” which does not exceed grinder capacity. Points 
on the line represent combinations which would use all grinder 
capacity, ranging from 20 standard and no deluxe models to 40 
deluxe and no standard models. 

Figure 2 shows the constraint on polishing capacity. As 
before, points below or on the second line represent combinations 
which are within or no greater than polishing capacity. These 
two constraints, together with the horizontal and vertical axes, 
produce an area of possible or feasible comoinations of S and 
”D” which do not exceed the capacities of either type of ma- 
chine. This area, shown cross-hatched in Figure 2, is called the 
feasible region. Also included in the region are combinations 
which fall on the boundaries. 

The next thing to do is. to take profits into account by re- 
ferring to our expression for profit of 3S plus 4D dollars. While 
it can be readily seen that any point in the feasible region ex- 
cept the origin (S=0, D=0) will result in a profit, we are look- 
ing for the point or points that represent the policy with the 

greatest profit. One way to approach this is by assigning the 

profit for any known feasible solution, such as $60.00 from the 

earlier policy of producing all standard models (S=20, D=0), 
to the right-hand side of our profit equation. This profit equa- 
tion, 3S + 4D = $60.00, defines the dashed line shown in 
Figure 3. Substituting any other vain ' for profit into the profit 
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Figure 1. Solution space with grinding constraint. 
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equation (3S ~f 4D = profit) would define another specific line 
parallel to this one. Remember our objective is to find the great- 
est profit. Geometrically, we do this by finding that profit line 
which lies as far away as possible from the origin of the graph 
and still touches the feasible region. For our example, we can 
draw by inspection the second dashed line (parallel to the first) 
shown in Figure 4 and then calculate the values for ft D” and 
"S 19 for the point of intersection of this profit line and the 
feasible region. This point of intersection tells us how many 
standard and deluxe models to make in a week. Substituting 
these values into our profit equation tells us what the maximum 
profit will be. The optimum solution of this problem shows that 
we can make $110.00 per week based on the manufacture of 10 
standard models and 20 deluxe models, as compared to the 
$96.00 possible by producing deluxe models only. 

Now extrapolate and consider the dental clinic model, which 
has many more equations than you have seen. Each equation 
contains an expression of the limitations of junior students, 
senior students, faculty, various clinical specialties, dental units, 
various clinical teaching areas, and the approximately 75 pro- 
cedures that are performed in our dental clinic, plus, of course, 
the time constraints within which they are performed. If you, can 
visualize such a set of equations, you will have a fairly clear 
conception of the linear programming model that we have been 
working with for the past three years. 

Obviously, when one gets more than five or six lines on a 
graph, it becomes very difficult to tell exactly what is going on. 
A problem containing more than three simultaneous equations 
isn’t easy. This is why the computer is essential to this kind of 
research. I think you might be impressed with the efficiency with 
which the machine solves the dental clinic problem. An early 
run of this problem included 73 such equations, with a total of 
113 variables. The entire operation required an investment of 
computer time of one minute and 49 seconds at a cost of $10.94. 

That is only one part of the total problem. We have never 
successfully run the complete problem for a number of reasons. 
Once we ran out of disk space. On another occasion, our job 
was cut off by a time limit of 99 minutes. At $350 per hour, 
you can see why we do not play around with this unless we 
mean business. Although we still have not run the full problem, 
we can, I think, make the statement that linear programming 
has been demonstrated to be a feasible management technique 
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Figure 3. Feasible region (enlarged scale) as defined in Figure 2, with 
profit equation. 
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Figure 4. Feasible region with optimal solution. 
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to use in clinical dental education. If it were not feasible, the 
machine would have said, "There is no feasible solution.” 

LINEAR PROGRAMMING AND TOTAL PATIENT CARE 

The other part of this story is total patient care. Earlier I 
told you that our project is a practical application of the linear 
programming problem. The clinical phase of this work was ini- 
tiated on a pilot basis in June 1968. It was aimed at performing 
several functions, the first of which was to provide an additional 
data base for our linear programming model. Most of the data in 
the model now is the result of guesswork. Secondly, we wanted 
to introduce automatic data processing to the students, the faculty, 
and our staff. Thirdly, we wanted to introduce a new curricu- 
lum, based on the comprehensive care concept, which we did 
in the fall of 1969. 

This phase of the project— total patient care— has involved 
five junior students to whom patients have been assigned 
manually because a computer program designed to match patients 
to students is not yet fully operational. The five students were 
selected by dividing the junior class into five academic levels and 
choosing one student at random from each level. Five more 
juniors joined the total patient care (TPC) group in the fall of 
1969. Patients were selected who would provide a balanced clini- 
cal experience equal to or greater than the usual clinical require- 
ments in the various departments. Progress of these students has 
been monitored constantly, and monthly reports have been pro- 
vided to department chairmen and administrators. A year-end 
report was developed in May 1969 and is used to compare 
these students with the rest of the class, which serves as a con- 
trol group. 

Because of the nature of the program, the TPC group did 
about 15 times as much preventive dentistry as the other stu- 
dents. Their experience in pedodontics is not included because 
there is not enough information. In oral diagnosis, the control 
group made more radiographs but did less charting. In endodon- 
tics, the control group did an averages of 3.6 root canals as 
compared to 2.8 for the TPC group; in' periodontics, the con- 
trols did fewer major cases, but more minor cases. 

In fixed prosthodontics, there is very little difference be- 
tween the two groups. In removable prosthodontics, the TPC 
group did only about half as much. In operative dentistry, which 
is perhaps the most important and certainly occupies a large por- 
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tion of the curriculum, the TPC group came out in front in gold 
foils, amalgams, inlays and total, while the controls accomplished 
more in the silicate/ resin area. Of course, these data do not 
show the total clinical activity of either group. The TPC group 
participated in only two block assignments— surgery and DAU. 
On the other hand, the controls did preventive work for only 
one patient while the TPC group did this for every patient. 

I leave it to you to interpret this data as you wish. We are 
evaluating the project in several ways, particularly by using the 
data gathered concerning the measurement of attitudes toward 
total patient care. Clinical ability and interpersonal relationships 
have not been evaluated as yet because there is insufficient in- 
formation. Some of the measures that we plan to use, and are 
using, include the quality, quantity, and speed of clinical pro- 
cedures, and scores on board examinations, both national and 
state, comparing the TPC and the control groups. 

We also plan a computer-based, interactive, diagnostic sim- 
ulation as an evaluation instrument. This scheme, we think, will 
be the crux of the evaluation because we are trying to train bet- 
ter diagnosticians. 

STUDENT AND INSTRUCTOR WAITING TIME 

There is another phase of Operations Research that is per- 
tinent to this discussion. Although queuing and work loading 
studies have not been integral parts of the project thus far, they 
are a logical extension thereof and I personally feel that they 
have a greater potential than linear programming. We have made 
limited pilot efforts by timing contacts between instructors and 
students in the clinic by quarter hours, as well as by monitor- 
ing down time, i.e., that time when an instructor has no de- 
mands for his services. Simultaneously, the length of the queue 
behind the instructor was recorded. 

As far as I have learned, there is no efficient, satisfactory 
method for acquiring this data at the moment, and therefore we 
are going to have to develop a method before we can proceed. 
Wo hope this kind of study will reveal patterns of instruction 
in clinical modes that would permit more efficient scheduling of 
Lculty, students, and physical facilities. The only thing I can 
say at this time is that a student who is waiting for an instruc- 
tor learns very little. And it is equally apparent to me that the 
instructor who is waiting for a student to demand his services 
does very little teaching. Both of these situations exist to a large 
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extent in clinical dentistry. A study of laboratory teaching will 
probably be even more fruitful since there are far fewer variables 
to worry about. 

EQUIPMENT 

Now I would like to mention the equipment with which 
we have been working. In the dental building there are four 
units of equipment, all rented from IBM. There is a 1051 con- 
trol unit which controls a 1052 printer keyboard and 1056 card 
reader. The fourth unit is an 029 key punch, which operates as 
a separate unit. The first three units form what is called a 1050 
data communication terminal. In earlier sessions of this confer- 
ence, I noticed that a number of speakers, in disucssing CAI, 
mentioned using this kind of terminal equipment. Ours is in the 
medium price range; less expensive and more expensive equip- 
ment are both available. The terminal is connected to the IBM 
360 Model 75 computer at the Triangle Universities Computation 
Center, which is in the Research Triangle Park, 12 miles away. 

PROBLEMS 

You might also be interested in knowing some of the prob- 
lems we have had. If I can enumerate some of them, and there- 
by save you from the same pitfalls, it will be worthwhile. Our 
chief problem has been time. The TPC students have pointed 
out to me that it takes more of their time to run a comprehen- 
sive care program than it does if they simply show up for block 
assignments and work on whoever is there. In other words, they 
are actively engaged in the mangement of a practice, and this 
takes time. Of course, the series of diagnostic procedures that 
they are required to do is considerably greater than that for the 
control group. In some ways their activities are more rigidly 
controlled. They have less freedom to decide when and for whom 
they work. That is, they have to satisfy the patients’ demands 
and these are more stringent than block assignments. 

The TPC students feel that with their experience under the 
new system they are better prepared to enter private practice 
than they would have been otherwise. However, spending so 
much time on diagnostic and management procedures, they have 
had less time to spend on restorative procedures. This places 
them at some disadvantage, compared to the control group, in 
equaling the minimum requirements. I am afraid that on this 
point there has been some intimidation on the part of the faculty. 
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Even though he does not have formal requirements in the TPC 
group, still a student had better do x number of procedures. 

Time has been a factor for me personally. I wish I could 
have spent more time counseling these students. It is my guess 
at this point that ten students is the absolute maximum that 
any one faculty member can supervise in a comprehensive pro- 
gram. Those of you who are responsible for hiring and firing 
and getting the budget to pay salaries will have to find more 
money and more faculty if you are considering comprehensive 
care. 

Another problem has been the maintenance of two record- 
keeping systems. This was necessary because the computer pro- 
gram was not ready. This was the result of a bad guess on my 
part and I will take full responsibility for it. We hoped that 
this program could be written in a period of three months. 
Eighteen months later, we are still writing. We hope we will 
not be more than two years off schedule. 

Yet another problem is keeping patients. We have had very 
good fortune, but there have been a number of drop-outs, as in 
private practice. Some do not like the program after they get 
into it and they leave. 

A further problem has been predicting, with a satisfactory 
degree of accuracy, the clinical needs of these patients. We do 
this on the basis of bite-wing radiographs, a Panorex film, and 
a cursory clinical examination. The major problem is in opera- 
tive dentistry. As you know, in preparing a cavity, many times 
you cannot tell what kind of filling material must be used in 
that tooth until you have removed all the decay and unsupported 
enamel rods. What starts out to be a Class I amalgam may end 
up as a Class II inlay. 

In addition to that, our operative department has a clinical 
grade based on quality and quantity, and while I might be able 
to guess fairly accurately how many surfaces are going to be 
required to restore a tooth, there is no way that I could tell 
what kind of grade the student is going to get on the restoration. 
Nevertheless, in scheduling patients we are expected to take work 
in progress, including grades, into account in making our assign- 
ments. 

Integration of the total faculty into the program is also a 
major objective of our project at UNC. The Curriculum Commit- 
tee has decided that beginning in the summer of 1970, the 
screening of patients will be done by the faculty. All cursory 
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examinations, selecting the patient, planning treatments, and 
counseling students, will be shared across the board with the 
entire staff. It will not be a function of just the oral diagnosis 
or the screening area staff. 

So much has been said about problems in keeping records 
for a computer-based system that I will simply say that we have 
those problems too. Those who are uninitiated should know that 
the handling of exceptions occupies about 90 percent of the ef- 
forts of data-processing people. There are some rough days 
ahead for you if you think the computer is going to eliminate 
your problems. 

In closing, I cannot pass up this opportunity to offer some 
philosophy. You have heard reports here of people using compu- 
ters for grading, record-keeping, instruction, and many other 
functions. If you have any administrative problems in these 
areas, do not expect the computer to solve them. More likely, the 
computer will compound them. Similarly, do not expect a compu- 
ter to save you any money. It is a machine that crunches num- 
bers; by itself it will not teach, it will not grade students, and 
it will not keep records. But it can help you do these things for 
a price— and it is a pretty steep price. 

Finally, I would like to say that I heartily support CAI and 
all the other computer uses that are being discussed here, but I 
am personally convinced that operations research techniques offer 
the greatest potential in planning to meet the challenge posed by 
the health care demands of the seventies and eighties. 
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I would like to start by speaking about clinical evaluation 
and then about a method of keeping track of the student’s pro- 
gress. At USC the primary problem seems to be relating quality 
of clinical operation with quantity. Some schools have a require- 
ment system, some allocate points for specific job operations, 
and some do both. We are currently able to keep track of qual- 
ity, in terms of grade point average, quantity, or the amount of 
clinical experience and, in addition, have developed a third 
measure which combines quality and quantity in what we feel 
is an objective overall measure of clinical performance. 

Quality of clinical performance is graded on a four-point 
scale. On any particular job operation, we suggest grading by 
deletion. Take, for example, an inlay which could be given a 
grade from 0.0 to 4.0. The job might be broken down into steps 
which are scored separately and totalled to arrive at a grade. 
Forty points might be assigned to the job as follows: the pre- 
paration may be worth ten points; the impression, five; the wax 
pattern, another five; and so on to completion, which is worth 
perhaps another ten points. If you want to be explicit, you can 
break down each and every item into line angles, flares, etc. At 
the end of the operation the instructor will then total the points 
and come up with perhaps a total of 32 out of 40 possible points 
for a grade of 3.2. 

In order to standardize our measure of quantity we have 
determined the average number of hours required to perform each 
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clinical operation. The inlay just described might take ten hours 
to complete on the average and would be assigned a value of 
ten baseline points. Baseline points for completed operations are 
accumulated within departments (and overall) to yield a measure 
of experience obtained regardless of grades. 

To retain a numerical measure of both the quantity and 
grade of a student’s work, while permitting both a requirement 
system and a total patient care system to operate, we developed 
what we call a translated baseline point system. Table I demon- 
strates how baseline points, or hours if you will, are combined 
nonlinearly with grades to yield translated baseline points. 



Table I 

Example of Conversion of Baseline Points 
to Translated Baseline Points 



Letter 


Operation 


Baseline 


Multiplication 


Translated 

Baseline 


Grade 


Grade 


Points 


Factor 


Points 


A 


4.00 


10 


3.000 


30 


B 


3.00 


10 


2.625 


26 


C 


2.00 


10 


2.000 


20 


D 


1.00 


10 


1.125 


11 


F 


0.00 


10 


0.000 


0 



Assuming that a student completes a job worth ten baseline 
points, if he receives a 3.0 grade, or what you might call a B, 
he would get 2.6 times as many translated baseline points as 
baseline points for the operation. Should he receive an A, or 
grade of 4.0, his multiplication factor would be 3.0 and he would 
get three times as many translated baseline points. These multi- 
pliers were selected by a faculty committee at the school to weight 
quantity of work slightly more heavily once the minimum grade 
of C is exceeded. The rationale is to ensure that even the top 
students receive a reasonable amount of experience in each de- 
partment. The multipliers in Table I are based on the following 
equation, in which Y is the multiplication factor and X is the 
grade for the operation on a 4.0 scale: 
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Y= 10 ( X) ~ (X) 2 
8 

We keep track of all students throughout their clinical ex- 
perience. Figure 1 shows the first of the three sections of the 
student’s copy of his progress report which he receives monthly. 

One of the reasons we furnish monthly progress reports is 
to minimize the anxiety and fear that students feel about evalua- 
tion as documented by the Guild Report.* Slow feedback of 
grades and class standing was cited as one of the main causes 
of their concern. Except for occasional computer delays, we get 
reports to the student fairly rapidly and we maintain, at all 
times, the reference point of his overall grade average. We have 
his current standing— that is, current from the last print-out 
to the present one — and we have the previous year’s. 

In order to understand Figure 1, look at F. PROS (Fixed 
Prosthodontics) in the department column. Here the student has 
a 2.70 grade point average and ranks fiftieth in the class on the 
basis of quantity. Notice, however, that in the next two columns 
he has 612 baseline points, and is ranked eleventh in class on 
the basis of quantity of work done. In the fifth and sixth col- 
umns we find that he has acquired 1523 translated baseline 
points resulting in a rank of twelfth in the class in this depart- 
ment — a composite of the other two rankings. 

Each department chairman, acting autonomously and at his 
discretion, decides what the minimum translated baseline point 
requirements will be for the purpose of determining final course 
grades. 

The last column, Percent Complete, shows the student just 
how close he is to completion in any one department at any 
time, according to the year’s total translated baseline point re- 
quirements as determined by the department chairman. 

Work in progress (as shown in Figure 2) is listed in order 
of starting date so that we can locate a particular case according 
to starting date. We have the patient’s name, the patient’s num- 
ber, and the serial number in our input form. (I will explain 
later how we get the input into the computer.) We have the 
number of baseline points that the item is worth, the tooth num- — 
ber, and the job code. Each and every dental operation is coded. 



*Guild, R.E. Questionnaire studies at three schools of dentistry. Journal of Dental 
Education , 1966, 30, 344-353. 
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Figure 1. Sample of monthly progress report provided each dentai 
student summarizing the clinical work he has completed by department. 
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Figure 2. Continuation of monthly progress report provided each den- 
tal student showing the clinical work he has in progress according to 
date treatment was "started. to 
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The work completed is printed out by department as shown 
in Figure 3. We are no longer concerned with the starting date 
because the job is completed. We are now concerned with the 
items that the student has finished, with his grades, the points 
he has received, and so forth. These are the items that are re- 
flected in the top line of Figure 1 to give him his overall aver- 
age and ranking. 

?4otice the Payrnnt Diff column in Figure 3. We have set 
up a system whereby the computer checks to see if the proper 
payment for any operation is made. An interesting sidelight 
here is that since going on this system our income in crown and 
bridge alone jumped $10,000 a year over our projected average, 
because of tighter control. Without the computer to cross-check, 
billing mistakes would go undetected. For example, a student 
might do a five unit bridge, write it up as a three unit bridge 
for the cashier, who may be too busy to check, and even though 
the instructor has signed off for a five unit bridge, we don’t 
get paid for two of those units. Under the new system, if the 
student writes it as a five unit bridge and the instructor verifies 
it, we are going to get paid for that five unit bridge. If a differ- 
ent fee comes in, it shows up here in the payment difference 
column. For those concerned with clinic budget and clinic in- 
come, here is an easy way to increase revenues. 

Figure 4 is the Department Status Report which each de- 
partment head receives. It shows all the work done in that 
department, by student. 

The computer input documents which I referred to earlier 
consist of a manifold form in five differently colored copies made 
up like a department store salesbook. The tGp white original is 
the " start” sheet which the student fills out and has the patient 
sign. (This form helps avoid those law suits in which the pa- 
tient says, "I didn’t know what you were doing— you were 
standing behind me and I couldn’t see.”) Once the "start” 
sheet is signed, the student then calls the instructor over to 
make a start check, and he is off and running. The instructor 
then tears out the "start” slip, puts it in his pocket, and sends 
his coat to the laundry. This happens! The point I am making 
here is that no ir\Cthod of inpul is any good unless you get 
people’s cooperation. In our case, it was up to the departmental 
secretaries to go through the pockets of the instructors’ gowns 
before they were sent to the laundry. The start check is working 
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Figure 3. Continuation of monthly clinic progress report furnished 
each dental student showing the individual cases he has completed. 
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well now, but it did take some time to get people used to work- 
ing with the forms. 

Upon completion of the operation, the student must have 
the cashier validate receipt of payment on the second yellow 
sheet which he then takes to the instructor who signs, grades, 
and dates it. The instructor then removes it from the book. The 
data from this sheet appears in the print-out shown in Figure 3 
and Figure 4. The cashier retains the fourth green copy and the 
patient receives his copy, the third blue one. It is very interest- 
ing that the phone calls regarding income tax we now get at 
year-end are few and far between. The student’s fifth pink copy 
remains in the book as his rtcord in case an error in key punch- 
ing occurs. We tell the student he is in deep trouble if he loses 
one of these books. So far, we have not been plagued with that. 

The only problem we found in coding our operations was 
in crown and bridge. There are an enormous number of combi- 
nations of restorations and types of bridges for which to pro- 
vide. With the liberal number of spaces we had left on our 
forms, however, we found that we could keep track of these up 
to and including a six unit bridge. 

This, then, is our method of keeping track of clinic stu- 
dents at USC and our method of evaluating them. We are in a 
trial period, at the moment, with a comprehensive patient care 
system. It will be modified to the extent that the junior year 
will use a requirement system and the senior year will be an 
internship year. We feel that if the patient assignment aspects 
are handled properly, we can surreptitiously have a requirement 
system within a comprehensive patient care program. By judi- 
ciously allocating patients to your students, you can be sure that 
they meet your basic requirement for the practice of dentistry 
within a total care package. 
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Part 2. REPORTS OF DISCUSSION GROUPS, 
and PROJECT ABSTRACTS 



INTRODUCTION 

On the basis of the interests and activities of conference 
participants, four broad areas of computer application in dental 
education were identified: (1) admissions and academic evalua- 
tion, (2) computer-assisted instruction, (3) dental school opera- 
tions and educational administration, and (4) student evaluation 
in clinical coursework. Discussion groups organized to discuss the 
state-of-the-art of computer applications in these four areas met 
three times each during the conference. 

Activities within the discussion group meetings were of two 
sorts. First, in an informal seminar atmosphere, each participant 
presented a summary of the work planned or in progress at his 
institution. Once these projects had been presented and discussed, 
the focus of attention was directed to the discussion of problems 
general to the group and to the identification of possible direc- 
tions for solution. At the conclusion of the conference, the dis- 
cussion group leaders reported the activities of their groups in a 
panel session before the entire conference. The remainder of this 
section contains both the reports of discussion group leaders as 
well as abstracts of the projects discussed during the meetings of 
the individual discussion groups. Reports and abstracts are or- 
ganized according to area of application. 

Some mention should be made of the nature and purpose 
of the abstracts. They are intended to provide the reader with a 
quick overview of the computer-based activities at a particular 
school. As preparation for the conference, participants were re- 
quested to develop written summaries of the computer-based den- 
tal education projects in which they were involved. The sum- 
maries served as a basis for discussion during the meeting and 
provided each participant with a permanent record of the activi- 
ties planned or in progress at other schools. To obtain economy 
of length and uniformity of format, the original summaries were 
abstracted for inclusion in this report. 

Capitalized keywords are used within the abstracts to signal 
areas of interest: PROJECT GOALS, IMPLEMENTATION, 
COMPUTER (HARDWARE) SYSTEM, PROGRAMMING LAN- 
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GUAGES, VOLUMES, SPECIAL RECORD FORMATS, STAFF, 
SPECIAL PROBLEMS and FUTURE PLANS. 

Hopefully the abstracts will assist dental educators and 
others in identifying projects of interest. Further information 
about a particular project should be obtained directly from the 
author of the abstract. The appendix provides the addresses and 
telephone numbers of participants as of May 1970. 



ADMISSIONS 
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ADMISSIONS AND ACADEMIC EVALUATION 



Report of Discussion Group 



Harry Weisenfelder, Chairman 

Leonard H. Kreit, Recorder 

In our meetings we concentrated on admissions, discussing 
some of the current activities and unmet needs, and were fortu- 
nate in having in the group representatives of three dental 
schools that have already made a significant start on develop- 
ment of computer systems for admissions. I believe the partici- 
pants were pleased to find there was a great deal of commonality 
in the approaches that they had taken. In all cases they are 
beginning their data collection with the admission procedure and 
will subsequently expand to clinical ratings. A further step is to 
collect, on a continuing basis, information on graduates to deter- 
mine if the individual who is now in practice is providing patient 
satisfaction. In this regard, Dr. Donald Strachan described a 
very interesting study being done at the University of Michigan 
which used patients’ appraisals of their treatment to determine 
how effective that school’s education program has been in meet- 
ing the needs of the dental public. 

We had a lengthy discussion on the desirability of promot- 
ing compatibility of data among schools and discussed using the 
term compatibility rather than uniformity. We would be happy 
to settle for compatibility. We decided that it would take a much 
larger group than our discussion group, perhaps a much larger 
group than is assembled here, to promote the compatibility of 
information systems that is needed. We hope that the present 
conference will provide a start in this direction. Perhaps the 
American Association of Dental Schools or a similar body could 
be of some help by providing a central clearinghouse function 
for existing procedures, or by calling together a group of experts 
to provide some guidelines and ground rules for compatibility. 

A valuable suggestion was made that it would be very help- 
ful for schools who do not yet have computer capabilities to 
nonetheless begin to maintain the information they are collecting 
in the area of admissions or student records (or the other data 
that they deal with continuously). If they could maintain this 
data in machine-readable form, it might later prove very valu- 
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able, even if at present it is as simple a system as keypunching 
directly from existing forms and storing the information for future 
use in a computer-based system. 

Next we came to a subject that deserves a great deal of 
attention— the desirability of having a central clearinghouse or 
application service to handle much of the routine work that is 
being done by all of the dental schools. One of the participants 
said his school is expecting costs of approximately $12,000 a 
year for merely transcribing certain aspects of the information 
provided by applicants. As the result of discussing such a ser- 
vice, the group developed a formal recommendation which I 
would like to read: 

The admissions and academic evaluation discussion 
group recommends that serious consideration 
be given to the establishment of a central service to 
assist dental schools in processing individual applica- 
tions for admissions. It was suggested that efforts be 
made to develop a system that would enable an indi- 
vidual to submit a single set of transcripts and letters 
of recommendation to a central source for distribu- 
tion in a uniform format to all schools to which the 
individual wishes to apply. 

The group thought that a pilot program involving a small num- 
ber of schools could be readily implemented, building on the 
experience of the centralized application service that is currently 
being utilized for medical schools by the Association of Ameri- 
can Medical Colleges. 

There were some additional suggestions on how this could 
be accomplished on a pilot basis using schools that are located in 
a center with a medical school. It might be appropriate to begin 
such a service at the time that potential applicants take the Den- 
tal Aptitutde Test (DAT). At the time they are completing the 
DAT forms they could also file permanent information to help 
establish a central clearinghouse for admissions information. The 
research potential alone is enough to warrant adoption of such 
a service without considering the savings in terms of time and 
money on the part of applicants and on the part of the schools 
in processing such information. There was considerable discussion 
of the value that this might have for the schools that are ex- 
periencing difficulty in getting either the quantity or the quality 
of applicants that they would like to have. Perhaps as a school 
approached its deadline it could report vacancies to the clearing- 
house. However, it was felt that this might not work as well, 
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perhaps, as a system whereby the individual’s application ma- 
terials might be used to indicate whether he was good potential 
for certain categories of schools, either by virtue of his location 
or other criteria which would eventually be developed out of anal- 
ysis of the collected information. We felt that this would result 
in larger pools of applicants to the schools and in some cases 
raise the qualitative level of applicants who are accepted. 

On a pilot basis, I believe that the American Association of 
Dental Schools might consider supporting this and I would like 
to see this proposal transmitted to the new standing committee 
on student affairs that will be responsible for admissions activity 
under the reorganization of the Association. 

In discussing the benefits of the conference, the group felt 
that they had benefited considerably by the opportunity to ex- 
change information with people working on similar problems, 
and they felt very strongly about having some mechanism for 
continuing contact. It was suggested that perhaps a central ad- 
dress or central office could be provided that would serve to 
collect and disseminate information of interest to people working 
in the computer area. It was also suggested that perhaps a sur- 
vey be made twice a year on ongoing activities, indicating the 
individuals involved, with emphasis on what the program is and 
providing sufficient information so that an inquirer could contact 
the proper individual for full details. There was also some dis- 
cussion of improving the usefulness of such a document by hav- 
ing people indicate honestly what problems they had experienced 
to help their colleagues avoid making the same mistakes. 

I think I have covered the highlights of the admissions 
group meetings. There was considerable discussion of desirable 
next steps once we have achieved the goal of establishing com- 
patible data bases in the schools and using them on a wide 
scale basis for admissions. One suggestion was to follow students 
after graduation in order to develop some better criteria than 
now exist for evaluating th^ effectiveness of dental education pro- 
grams. Evaluation, as far as the dental schools are concerned, 
now ends with the individual taking the National Board Exami- 
nation. With renewed interest in continuing education, and the 
possibility that relicensing will be in some way related to parti- 
cipation in continuing education, development of some evaluation 
criterion based on post-graduation performance will become more 
important in the future. 
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Project Abstract: UNIVERSITY OF ALABAMA 

Computer Assistance in Admissions at the 
School of Dentistry 

PROJECT GOALS were related to the fact that the number of applications for 
admission to the School of Dentistry was increasing, while the adequacy of selection 
procedures was decreasing. A small fact-finding committee, set up to review alterna- 
tive solutions, recommended complete investigation of computer use for development of 

1. a more efficient admissions system, and 

2. an information system that would provide data on students as applicants, 
scholars, clinicians and alumni. 

IMPLEMENTATION was accomplished some three years later when Phase I of 
the project— the admissions system— became operable. Basically, the system involves 
coding and entering applicant information, subjecting this information to the pro- 
gram which performs a number of evaluative operations, and distributing this data 
in print-out form to the members of the Admissions Committee. 

Phase II of the project is concerned with recording grades and class standing. 
Date collection began with freshmen students and is expanding by adding successive 
entering classes. After four years, all levels will have computerized records. Such 
data is made available each quarter to promotions committees and to students. 

Recording and evaluating students’ progress in clinical coursework constitute 
Phase III. Appropriate programs have been written and are undergoing tests. The 
system should be operational in the near future. 

STAFF includes a secretary-c-ider in the admissions office and a programmer 
at the Computer Center. The staff of the Computer Center is available on a con- 
sulting basis. 

SPECIAL PROBLEMS have been encountered in dealing with multi-departmen- 
tal responsibilities of staff members, a situation that contributes to work slowdown. 
Programmers are in limited supply and many have left the school to accept better 
positions. Considerable effort has been expended in clarifying the objectives of the 
computer-assisted admissions system for the benefit of faculty members and admin- 
istrators, i.e., assuring them that the system does not bypass the Admissions Com- 
mittee, which in fact makes all final selection decisions. Exchange of electronically- 
processed information with other schools has been hampered by differences in cod- 
ing, hardware, etc. 

FUTURE PLANS call for the introduction of Phase IV which would provide 
information on alumni. Data collection would begin in several years with the first 
group of graduates whose records are entirely computerized. Records of earlier grad- 
uates whose records are entirely computerized. Records of earlier graduates would be 
worked into the system in stages. Exchange of data with other schools and with 
professional associations is also contemplated. Work is planned in the development 
of translations and conventions that would enhance compatibility between systems. 

Clarence E. Klapper, Ph.D. 

Chairman, Admissions Committee 
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Project Abstract: UNIVERSITY OF TEXAS 

Acquisition of an Electronic Data Processing 
System for the Dental Branch 

PROJECT GOALS revolve around improved control of information regarding 
applicant selection and student progress, as well as more effective management of 
patient records. Production o summary reports on operations in the Dental Branch, 
handling of large numbers of patients and transactions, and formulation of an opti- 
mal response to changing clinical techniques have contributed to the need. 

IMPLEMENTATION has been broken down into five phases. Phase I is con- 
cerned with applicant selection. Various reports and evaluations of applicants, in- 
cluding ranked lists, are electronically generated and presented to the Admissions 
Committee. 

Phase II involves the handling of student records, including transcript evalua- 
tion, course grading and averaging, and preparation of final transcripts. 

Phase III deals with management of clinical patient accounts and students’ 
records regarding the patients. A standardized clinical grade card and a uniform 
number code for all clinical work have been devised. 

Phase IV includes business operations and assisting the Business Office in sys- 
tems preparation. 

Phase V activity is related to curriculum construction. Along this line, code 
symbols corresponding to the logical sequences of a teaching topic have been de- 
veloped and have proved to be useful in curriculum scheduling. 

The COMPUTER (HARDWARE) SYSTEM is composed of the IBM Model 7094 
located at the Common Research Computer Facility of the Texas Medical Center. 
An IBM Model 360/50 facility at Baylor Medical College i9 utilized by means of 
an IBM Model 2740 Remote Terminal using leased line connection. An IBM Mod^l 
2760 Optical Image Unit is on order. 

PROGRAMMING LANGUAGES in use are FORTRAN IV for the IBM 7094 
and PL/1 for the IBM 360/50. Programs originally written in COBOL have been 
translated to PL/1. 

VOLUME of data processing is large in view of the fact that there are approx- 
imately 550 students at the Dental Branch and roughly 10,000 new patients making 
75,000 visits annually. 

The STAFF of the Data Processing Department includes a systems analyst- 
programmer, a senior keypunch operator-programmer and a secretary. 

A SPECIAL PROBLEM lies in the lack of direct acce 99 to data in storage. 

FUTURE PLANS provide for direct access entry to the systems, specifically by 
means of remote terminals for use in active input areas. 

Donald C. Kroeger, Ph.D. 

Chairman, Pharmacology 
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COMPUTER- ASSISTED INSTRUCTION 



Report of Discussion Group 

Richard S. MacKenzie, Chairman 

Virginia G. Sturwold, Recorder 

I would like to review our activities in two parts. First, I 
want to give you a quick idea about some of the informal pre- 
sentations that were made concerning what is going on in com- 
puter-assisted instruction, and secondly, I want to touch on two 
problems that we discussed at length in our three meetings. 

Presentations were made of CAI work at four different 
schools. The first was by John Weber of Cornell, who described 
an anatomy program which operates on the IBM 1500 system. 
The particular program covers several topics, but the one we 
discussed was teaching the function of the eye muscles. Cornell 
found that experts in anatomy could complete this program in 
about three minutes, whereas a novice might take two hours. 
They also tested the retention of the novices by giving them the 
program again after some time had elapsed and they found that 
the average time to go through the program a second time had 
dropped to about ten minutes, which indicated that some type 
of learning was clearly going on. One problem that was brought 
out in the discussion of this program was that it is extremely 
difficult for a single school to spend the amount of time necessary 
to develop a full-grown computer-assisted instruction program. 
Cornell has produced so far only about ten hours of instruction; 
presently they are questioning whether they should allocate the 
additional time and resources that will be necessary to develop a 
complete program. One solution Dr. Weber presented is approach- 
ing the medical schools at their annual meeting and asking for 
cooperation from the anatomy departments around the country. 
In this fashion one might gain the cooperation needed to produce 
a program of approximately 100 hours. 

Another problem which we discussed was a plea that those 
of us who are involved in computer- as sis ted instruction should 
spend more time attending to the problem of behavioral or teach- 
ing objectives. That is, we should pay some attention to how 
these objectives, which are really intermediate objectives, fit in 
with achieving the total terminal objectives of the program. This 
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amounts to questioning the validity of intermediate objectives in 
relation to the final products that we are trying to produce. The 
response to this was that the people who developed the objectives 
were experts in anatomy and the particular programs that have 
been developed have tied anatomic concepts to relevant clinical 
problems. This is one of the approaches at Cornell for linking 
intermediate and terminal objectives. I think the plea is one 
that deserves attention. 

The second program we discussed was one that has been 
running at Stony Brook in New York. It was produced by Drs. 
Malcolm Skolnick and Mortimer Shakun and is one in which 
the computer scans radiographs and identifies diagnostic features 
of the radiograph. Topics of discussion were the problems of 
development, the program’s potential in the diagnostic training 
of dental students, and its possible application not only to the 
reading of radiographs but also to such things as the screening 
of blood cells and the scanning of histological-pathological sec- 
tions. 

The third CAI program we discussed was done here in San 
Francisco at the University of California Medical Center and was 
demonstrated for us by Drs. Martin Kamp, Harvey Brody, and 
Richard Rozen. It has the general purpose of teaching sophomore 
dental students interview techniques useful in diagnostic prob- 
lems, and employs the PILOT language developed by Dr. John 
Starkweather with emphasis on natural language processing 
capabilities. 

The fourth program was the work of Dr. Devore Killip at 
Iowa. Their CAI demonstration project brought to light interesting 
problems, one of which was attitudes — the unwillingness of fac- 
ulty to use computer-assisted instruction at all. The strategy 
used at Iowa was to get the faculty together and give them a 
two-hour course each week for a semester. This allowed the fac- 
ulty to become well acquainted with computer-assisted instruction 
before making a decision regarding whether Iowa should go in 
this direction. After fifty-four faculty members had spent this 
time reviewing educational techniques and CAI, they decided 
that CAI was the direction they wanted to take. I think this is 
a very interesting approach. Unfortunately, it brings up the ques- 
tion of how they will be funded so that they can pursue the 
decision they made. This is obviously a question of general 
interest because CAI is so expensive. 

Once we had finished with these informal presentations the 
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group turned to discussing two problems. The first had to do 
with the amount of time and effort it takes to produce CAI ma- 
terials, and with what can be done to help expedite the sharing 
of materials. For the short term, it was felt that this conference 
is itself serving the function of allowing us to know one another 
and become aware of ongoing programs so that we can com- 
municate with one another. For the long run, we were interested 
in finding some way of sharing programs. Clearinghouses for 
information and depositories for finished programs were discus- 
sed. It was pointed out that Congress is presently holding open 
hearings on instructional technology with attention being paid 
to the problem of compatibility of machinery, computer lan- 
guages, and so forth, and that these hearings may eventually 
result in sharing material among schools. 

The second problem was the attitude of policy makers that 
is revealed in their frequently-asked question, "Isn’t computer- 
assisted instruction just a glorified electronic device for turning 
pages?” Although some CAI work has been no better than a 
page-turning device, our group feels that its potential is much 
greater than that, particularly in view of the shortage of faculty 
members, the growing shortage of certain types of teaching pa- 
tients, and the limitations on the types of learning experiences 
available for students. All these deficits can be overcome to some 
extent by a computer. For example, simulation of emergency 
situations by a computer is as possible as simulation of the 
routine parent by a manikin. Both uses will allow beginning 
students to learn things that will transfer very readily to the 
clinic. Simulation is one area, clearly, in which a computer 
can be used. Not to be overlooked is the potential of the com- 
puter for individualizing instruction because the computer has the 
capacity to adapt instruction to the student’s background and 
ability. This is very important in health manpower training 
because it will reduce the training time required. The important 
point was made that the computer affords us an excellent way 
to study human learning in certain areas of instruction that we 
have not yet been able to touch with our conventional teaching. 
Although development costs are formidable, studies indicate that 
the costs of CAI in the long run are competitive with other 
types of instructional systems. 

Finally, the point was made that by doing our experiment- 
ing now . with computer-assisted instruction we will be able to 
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guide the technology explosion in such a way that the hardware 
being developed will help in the teaching of dentistry in the 
future. 
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Project Abstract: CORNELL UNIVERSITY MEDICAL COLLEGE 
Research Directed Toward Increasing the 
Tutorial Capacities of the Computer in the 
Anatomical Sciences 

PROJECT GOALS include: 

1. Development of a pedagogical format for computer tutorial teaching in the 
basic medical sciences. 

2. Increasing the technical competence of the computer as a tutor. 

3. Simplification and improvement of the course preparation process. 

While emphasis has been placed on increasing technical systems capabilities 
rather than on developing a mass of course material, some experimental programs 
have been written for the anatomical sciences. 

IMPLEMENTATION in the first area consists of a branched conversational dis- 
cussion program with no fixed beginning or end. A starting point is selected either 
by the student or by the computer. Correct answers to clinical problems associated 
with the discussions lead the student in a straight line from one discussion to the 
next. Responses and degree of comprehension are recognized and measured by means 
of key words and questions. Failure to answer a key question draws the student 
down into a branched discussion network. 

Technical improvements in the second area involve: 

1. A natural language approach based on key word analysis, along with pre- 
scanning in initial response recognition. 

2. Artificial intelligence, i.e., incorporation into the program of the ability for 
modification of instruction based on " experience” with students. 

3. Interactive graphics, providing the student with the ability to respond to 
detailed anatomical drawings on the face of a cathode ray tube (CRT) using a light 
pen to identify structures and to "draw” on the CRT face. 

Finally, course authors are being assisted in preparing graphic materials for 
display by the development of a computer- con trolled optical scanning system cap- 
able of coding a detailed picture in machine- readable form automatically. A further 
aid is provided by an author-interrogation program which takes the author step-by- 
step through the complicated decision-making process associated with writing a course 
program. 

The COMPUTER (HARDWARE) SYSTEM consists of an IBM 1500/1130 sys- 
tem. PROGRAMMING LANGUAGES include COURSEWRITER II and 1130 assem- 
bler language. 

STAFF assistance has been provided by two systems programmers and an 
illustrator. Student participation and contributions to the authoring and program- 
ming have been significant. 

Support for the project is provided by Grant #50/68 from the National Fund 
for Medical Education. 

John C. Weber, D.D.S. 

Assistant Professor, Anatomy 



Wilbur D. Hagamen, M.D. 
Associate Professor, Anatomy 
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Project Abstract : OHIO STATE UNIVERSITY 

Computer- Assisted Instruction Experience 
at the College of Dentistry 

PROJECT GOALS are: 

1. To design and use computer-assisted instruction (CAI) as a supplement to 
classwork and for simulation of actual practice experiences. 

2. To ascertain user attitudes toward CAI. 

3. To evaluate the usefulness of CAI in meeting informational needs for con- 
tinuing education of health professionals at the community level. 

IMPLEMENTATION took the following forms. Two CAI programs were written, 
one for a course in Nutrition and the other a continuing-education course for den- 
tists and physicians, entitled "Oral Cancer Recognition.” The 10 to 14-hour Nutri- 
tion course is suitable as tutorial self-evaluation. The author, a faculty member of 
the Department of Biochemistry, College of Medicine, divided the course into eight 
topical areas: energy, carbohydrates, fats, etc. 

The 4 Vz hour "Oral Cancer Recognition” course is the product of members 
of the dental faculty and is designed to evaluate CAI in terms of the needs of con- 
tinuing professional education. The authors included six clinico-pathologic, conference- 
type cases in the course. Design strategy permitted user simulation of history-taking, 
physical observation and laboratory tests upon which to base diagnoses. 

User attitudes toward CAI have been evaluated by means of an on-line compu- 
terized interview and by means of a standard questionnaire. 

Of interest was the attempt by the CAI Study of the Ohio State Regional Med- 
ical Program (OSRMP) to train subject matter specialists as course authors. However, 
absorption of theories of programmed instruction and study of a programming lan- 
guage by such persons apparently hampered their writing efforts, possibly contribut- 
ing to delay in program completion. A team approach to coursewriting is now 
recommended, utilizing the services of an author consultant (CAI language special- 
ist), technical editor, input coder, and systems analyst. 

The COMPUTER (HARDWARE) SYSTEM consists of a time-shared IBM 
360/40 disk operating system linked by data telephone lines to two IBM 1050 
typewriter terminals in the dental building (as well as to 10 community hsopitals 
in central Ohio). In addition, the College of Dentistry is now linked with the 
main campus Computer Center’s IBM 360/50 through an IBM 2741 teletypewriter 
terminal. 

PROGRAMMING LANGUAGES include CPS and COURSEV r RITER III, Ver- 
sion 2 (IBMB). A translation program between the two computer systems is now 
operational. 

The system utilizes as a SPECIAL RECORDS FORMAT a random-access 35- 
millimeter projector and rear projection screen for visual presentations. 

Because this project is a cooperative effort, STAFF includes personnel from the 
College of Dentistry, the College of Medicine and the OSRMP-CAI Study. 

SPECIAL PROBLEMS were encountered in scheduling user time and in oc- 
casional system failures, due to excessive job demands on the time-shared hardware. 
Language translation between computer systems was another source of difficulty. 

FUTURE PLANS are evolving around a major curriculum change in the Col- 
lege of Dentistry by 1971, coinciding with an enrollment increase of 50 freshmen. 
A faculty education program to acquaint instructors with CAI and a learning-labora- 
tory based on CAI are contemplated. It is hoped that a program of simulated 
patient-management situations can be developed as well. 

Daniel E. Tira, Ph.D. 
Education Specialist 
College of Dentistry 



Gerald W. Gaston, D.D.S. 

Project Director, The Ohio State 
Regional Medical Program, 
Computer-Assisted Instruction Study 
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Project Abstract: STATE UNIVERSITY OF NEW YORK AT 
STONY BROOK 

Automated Scanning Analysis of Dental X-Rays 
Using the Clinical Decision Tree 

PROJECT GOALS are: 

1. Design and implementation of a hardware-software system cap- 
able of rapidly analyzing panoramic dental X-rays. 

2. Implementation of diagnostic procedures using the obtained 
data to produce a clinical treatment plan and accompanying cost esti- 
mate, consistent with information on the Panorex X-ray. 

3. Development of materials to facilitate instruction in X-ray read- 
ing, identification of significant radiographic data, diagnostic decision- 
making and treatment planning. 

IMPLEMENTATION of the system involved development of a cath- 
ode ray tube flying spot scanner interfaced to an IBM 1130 computer 
through an analog-to-digital converter. This COMPUTER (HARDWARE) 
SYSTEM, operating under program control, incorporates an optical 
system which reads the optical density from a selected spot on the 
X-ray. This information is returned to the computer and subjected to 
the pattern analysis algorithm. 

Analysis of the scanned X-ray is effected by a succession of binary 
logical choices incorporating an OVERALL scan to locate absence or 
presence of teeth, REGIONAL scans to determine relationships between 
teeth and to check properties of a whole quadrant, and LOCAL scans 
to determine properties of individual teeth. 

The clinical decision tree is the combination of the set of 9 'bi- 
nary ** questions about clinical properties of the mouth and the pre- 
scribed rules for proceeding to a new question based on previous 
answers. We will provide paradigms based on clinical requirements 
which examine individual data entries from the scanner on tooth or 
root location, tooth shape, density differences that indicate lesions or 
fillings, etc., and proceed on the basis of these findings to develop a 
profile for the particular patient. This profile is of course based on the 
particular path through the tree and is the sum of the individual de- 
cisions taken as a result of the working process. 

PROGRAMMING LANGUAGES include IBM-provided software, in 
particular FORTRAN, in which a number of recognition operations 
and scanner controls have been written. 

STAFF presently includes a mathematician, a communications 
scientist, a dentist specializing in radiography, a dentist-operations re- 
searcher, a data processing manager and an educational programmer. 

SPECIAL PROBLEMS have been encountered in eliminating ex- 
traneous scanned data without affecting pertinent data and analysis 
capability, careful positioning of the head during the X-ray process, 
recognition of head and mouth positioning, control of development of 
the X-ray, and normalization of the film gray levels to achieve inde- 
pendence of the exposure and development process. 

Most of these problems have been resolved by the use of a bite- 
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block with embedded metal objects of known properties. Images of the 
metal objects on the resulting radiograph, which is a SPECIAL REC- 
ORD FORMAT, provide fiducial marks for the orientation and calibra- 
tion of the radiographic image, and also provide a means of physically 
separating the images of the upper and lower teeth. A rapid statistical 
sampling procedure of the range of densities on a given film is per- 
formed at the beginning of the scanning process to supply information 
for the establishment of density thresholds for use in the analysis of 
the given film. The optical levels returned by the scanner may be nor- 
malized according to the observed opacity of the metal parts in the 
bite-block. This eliminates most of the difficulties inherent in variations 
of film exposure and development. 

FUTURE PLANS call for teaching X-ray interpretation through 
the use of CAI materials. The clinical decision tree will be used in 
selection of appropriate course sequences. The goal of this portion of 
the effort is to facilitate a student’s comparison of his analysis of an 
X-ray with the decision tree prescribed as part of the scanning process. 
Elucidation of the decision tree is thus seen as extremely useful. It may 
be used as a foundation to produce paradigms for general instruction. 
Coupled with the automated scanning process the decision tree may be 
used to derive an instructional review for an individual patient. 

The project is supported by the U.S. Army Research and Develop- 
ment Command. 



Malcolm H. Skolnick, Ph.D. 
Director, 

Biomedical Communications 



Mortimer L. Shakun, D.D.S. 
Clinical Investigator, 

Dental X-ray Computer Project 
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Project Abstract: UNIVERSITY OF CALIFORNIA, SAN 



FRANCISCO MEDICAL CENTER 
Computer-Simulated Patient for History-Taking 
at the School of Dentistry 



The PROJECT GOAL is to develop a simulated patient interview, in which the 
computer acts as the "patient.” The student, in "conversation” with the patient, 



A shortage of clinical patients and drawbacks of present teaching formats increase 
the need for an alternative method. 

IMPLEMENTATION of the Chief Complaint, History of the Present Illness, and 
Medical History sections of the interview has been accomplished. 

The interview itself is conversational in nature and can develop along a num- 
ber of different lines depending upon the questions asked by the student. Formula- 
tion of the program includes the following operations: 

1. Definition of the essential elements of the history. 

2. Agreement on a minimum amount of information necessary to make a diag- 
nosis. 

3. Development of a design strategy corresponding to the decisions previously 
made regarding the history and diagnosis. 

4. Storage, testing and modification of each program section as it is completed. 



The COMPUTER (HARDWARE) SYSTEM is located at the Computer Center, 
San Francisco Medical Center. Actual hardware includes the IBM Model 360/50 with 
an IBM Model 2741 Remote Typewriter Terminal connected by means of telephone 
access. 

Specialized PROGRAMMING LANGUAGES have been devised. The first one 
was COMPUTEST which allows unrestricted interactional sequences between student 
and computer. PILOT, or Programmed In struct i° n: Learning or Teaching, an out- 
growth of COMPUTEST, is currently in use. 

Television taping of student/* 'patient” interviews for evaluation is a SPECIAL 
RECORD FORMAT. 

Project STAFF consists of several members of the dental faculty, personnel from 
the Instructional Division of the Office of Information Systems, U.C.M.C. and a 
staff member of the Dental Health Center, U.S.P.H.S. 

FUTURE PLANS involve expansion of present operations with a view toward 
further improvement of student performance by freeing clinical faculty members to 
help students in specific problem areas. An interdisciplinary course in interviewing 
is also envisioned. 

Harvey A. Brody, D.D.S. Martin Kamp, M.D. 

Assistant Professor, Chief, Instructional Division 

Oral Biology and Dentistry Office of Information Systems 

Richard D. Rozen, D.D.S. Luigi F. Lucaccini, Ph.D. 

Specialist, Oral Biology Research Psychologist 



must obtain a history and make a diagnosis. The simulated patient is designed to 
improve the generally poor translation of lecture-learned skills into clinical practice. 



5. Student evaluation via an automated questionnaire and a modified checklist. 



Division of Dental Health, NIH 
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Project Abstract: UNIVERSITY OF IOWA 

Exploratory Study of Computer-Assisted 
Instruction in Dentistry 



The general PROJECT GOAL involves investigation of the potential of computer- 
assisted instruction (CAI) for presentation of dental subject matter content. The efforts 
of specialists in dentistry, education and computer science were combined in this 
case to achieve the following objectives: 

1. To gain experience in the use of the IBM 1500 system. 

2. To familiarize a selected number of the dental faculty with CAI. 

3. To prepare a CAI demonstration program in a dental subject matter area. 

4. To motivate the entire dental faculty toward course preparation for automated 
instruction. 

5. To determine the feasibility of a major effort in CAI at Iowa. 

IMPLEMENTATION of two main activities was undertaken concurrently. The 

orientation and training of dental faculty was accomplished through weekly seminars 
over a period of three months. Discussion topics included educational theory, pro- 
gramming techniques, computer technology, the IBM 1500 system and the COURSE- 
WRITER II language. 

At the same time, a demonstration course in Periodontics was being written 
for CAI. An expert consultant in Periodontics and a teaching member of the depart 1 
ment were designated senior author and junior author respectively. The junior 
author was responsible for the teaching format (sequence) of the course while the 
senior author was responsible for its content. The demonstration course included a 
two-hour sample unit of instruction in addition to introductory sections. The attempt 
to accommodate various student readiness levels in terms of the branching capabili- 
ties of CAI proved to be quite a challenge for the authors. 

Finally, participating faculty members tested the demonstration course during a 
field trip to Science Research Associates (SR A) in Chicago, the location of the near- 
est working IBM 1500 system. Subjective reaction to the CAI experince was favor- 
able. Thus far, no effort has been made to student-test the system. 

The COMPUTER (HARDWARE) SYSTEM consisted of the IBM 1500 System 
available at SRA, Chicago. This particular unit was selected because it offered great- 
er versatility than systems available on the Iowa campus. 

The PROGRAMMING LANGUAGE was COURSEWRITER II. The program 
itself incorporated as many types of branch loops as could be devised at the time. 

SPECIAL RECORD FORMATS included use of anatomical graphics and the 
Cathode Ray Tube (CRT) light pen for recognition of appropriate structures. 

STAFF consisted of members of the dental, education and computer science 
faculties. This group designed and conducted the weekly seminar and consulted in 
the preparation of the demonstration course. Members of the group included the 
project director, the senior and junior authors, a programmer and an audiovisual 
technician. 

SPECIAL PROBLEMS resulted from lack of experience with the 1500 system 
and took the form of language and hardware difficulties. Problems in course reduc- 
tion for programming were encountered as well. 

Devore E. Killip, D.D.S. 

Director, Division of Education Resources 
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Project Abstract: UNIVERSITY OF PITTSBURGH 



Computer-Assisted Instruction in Diagnosis 
at the School of Dental Medicine 



PROJECT GOALS are to provide a comprehensive assortment of 
learning experiences in diagnosis together with diagnostic services for a 
certain pool of patients needed in the clinical program. 

Too few patients have been assigned to dental students to allow 
them to achieve the desired competency in diagnostic skills. In addition, 
a change in clinical curriculum toward total patient care has caused even 
fewer patients— and therefore fewer diagnostic experiences— to be assign- 
ed to students. The new plan calls for patient experiences in the first 
year. However, the number of available diagnostic patients has not been 
commensurate with either first or second year needs. Clearly an alterna- 
tive source of diagnostic experience and service is necessary. 

IMPLEMENTATION of the project has not yet taken place. How- 
ever, procedures have been developed and are ready for programming. 
The essential feature of the program will be complex analysis capability 
to enable students to probe deeply into a selected number of patient 
histories. Students will take a history, in standard-sheet form, and will 
do a simulated physical examination as part of the in-depth problem 
solving/problem posing process. 

The COMPUTER (HARDWARE) SYSTEM includes an IBM 
360/50 located at the University Computer Center, and two IBM 2741 
terminals and one IBM 1052 terminal in the dental school. 

A SPECIAL PROBLEM is the lack of financial support, which 
has thus far prevented implementation of the project. 

FUTURE PLANS include program expansion to provide: 

1. Experience in treatment planning. 

2. Adjustment of various aspects of the operations in accordance 
with observed results. 

3. Improvement of the mechanisms whereby learning pace can be 
regulated individually. 

4. Contribution to and participation in an inter-university bank 
of programmed cases. 

Roy T. Durocher, D.D.S. 

Director of Clinics 
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Project Abstract: DENTAL HEALTH CENTER 



Development of an Automated Dental Training 
Simulator 



The PROJECT GOAL is to develop a full-form, computer-controlled 
dental training simulator. The manikin-like device would replace such 
items as "tooth simulators,” etc., and would provide a closer corre- 
spondence between clinical and laboratory learning experiences in 
operative dentistry. 

Actual IMPLEMENTATION of the project has not yet taken place. 
However, preliminary investigations are being conducted. The design 
of the desired simulator is expected to be very much like that of SIM 
ONE, a medical training manikin developed jointly by Sierra Engineer- 
ing Company, Aerojet-General Corporation and the University of South- 
ern California. The apparatus has been used to advantage in training 
anesthesiologists. 

In dentistry, skills learned in certain pre-clinical laboratory situa- 
tions bear little resemblance to those needed in clinical practice. In 
many instances, students must unlearn laboratory skills when they 
enter clinical training. Currently employed pre-clinical teaching devices 
often lack significant characteristics present in patients. For this rea- 
son, the full-form dental training simulator will be equipped with com- 
plete face, oral cavity, tongue, saliva, limited jaw movement and sen- 
sory mechanisms to register pain and heat. 

Project STAFF is associated with the Professional Education 
Branch of the Division of Dental Health, BHME, NIH, San Francisco, 
California. 

FUTURE PLANS involve using the results of feasibility and capa- 
bility studies to design a workable model of the simulator. A second 
phase consisting of product testing, and redesign, if required, is con- 
templated in conjunction with the first phase. Incorporation of a com- 
puter for purposes of manikin control, as well as to record performance 
will constitute a third phase of operation. The possibility of operating 
several simulators from one general purpose computer will be con- 
sidered. 

Donald F. Carter Luigi F. Lucaccini, Ph.D. 



Design Engineer 



Research Psychologist 



Dale W. Podshadley, D.D.S. 
Chief of Branch 



Professional Education Branch 
Division of Dental Health, 
NIH, San Francisco 



1G2 




DENTAL SCHOOL 
OPERATIONS AND 
EDUCATIONAL 
ADMINISTRATION 



163 



DENTAL SCHOOL OPERATIONS AND 
EDUCATIONAL ADMINISTRATION 



Report of Discussion Group 

Clifton E. Crandell, Chairman 

James M. Barnett, Recorder 

I have organized our discussions into three sections— first, 
a review of current usage of computers in dental school admin- 
istration; second, a discussion of hardware, software, and admin- 
istrative problems; and third, identification of some areas that 
we feel need to be explored in the future. 

The current activities we talked about are so numerous that 
I can only list them. We discussed billing, test scoring, averag- 
ing of grades, computing class standings, use of factor analysis, 
test item pools and item analysis, scheduling of students, sched- 
uling of space, matching and assigning patients to students, 
maintaining patients’ records, preparing form letters, problems of 
recording clinical achievement, grading faculty performance, al- 
location of resources, maintaining supply inventory, purchasing, 
patient followup and recall, grading for counseling purposes, 
handling student admissions, and information retrieval. It was 
observed that the number of dental schools employing computer- 
ized recordkeeping operations is large and growing. It was felt 
that most of these operations are executed satisfactorily although 
manual or non-computer methods are still widely used. 

In discussing hardware, it appeared that in general it is 
probably better to rent than to own. On the other hand, a 
thorough systems analysis must be done first, including a clear 
and precise specification of output, before an appropriate de- 
cision can be made about hardware. It was suggested that tabu- 
lating equipment might be more desirable than computer equip- 
ment for many of the administrative functions we discussed. An 
extension of this system selection problem includes consideration 
of time-sharing systems versus batch processing, and stand-alone 
versus remote terminal systems. Input forms that were mentioned 
as being currently in use included mark sense cards, optically 
scanned forms, Hollerith cards, port-a-punch, and manual 



16 4 



DENTAL SCHOOL OPERATIONS 159 

methods. One administrative problem discussed was shifting 
from grades to ether motivational mechanisms such as a flexible 
curriculum with acceleration features that would permit a student 
to graduate in three years or less. Another problem was ways 
and means of getting cooperation from faculty and students in 
implementing an automated system. The problem was recognized 
as important, but solutions were not clear. 

In proposing directions for the future, two general areas 
were touched upon— the activities that are needed, and the per- 
sonnel necessary to carry them out. If, was agreed that a systems 
approach to the problem of dental school operations is desirable. 
Even if a systems analyst does not generate a better solution, 
the administrator will have the benefit of a critical review on 
which to base his decision. 

Payment systems were considered from several aspects— pre- 
payment schemes, postpayment, third party collections, and 
billing procedures all received attention. The possiblity of basing 
clinical fees on time or difficulty rather than on piece-work fees 
was also discussed. It was observed that requiring the student 
to collect the fee was demeaning and of no educational value. 

A management information system is needed in each dental 
school to provide data on the cost of operation, cost of services, 
and utilization o£ facilities. Such a system would provide valuable 
input into decisions about contracts for services under insurance 
or prepaid treatment plans. Considerable discussion was gen- 
erated about the qualifications and administrative relationship of 
the person or group of persons who directs the data processing 
effort. Should the position be filled by a dentist with special 
training, or by an information specialist with dental training, or 
should it be managed on a cooperative basis by members of 
each discipline? We did not reach a decision on this point; in 
practice, available funds and personnel will probably influence 
this decision in the individual insitution. The group agreed that 
the director should report directly to the highest level of admin- 
istration. 

The establishment of a formal system for future exchange 
of information such as a newsletter or additional conferences is 
certainly needed. The outlook for the exchange of programs will 
be more favorable when they are machine-independent. The 
sharing of resources in the future appears to be essential because 
of the high costs involved in program preparation. 
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Project Abstract: HARVARD UNIVERSITY 
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Computer-Assisted Record Systems for Clinical 
Projects at the School of Dental Medicine 



The PROJECT GOAL is the development of automated systems for data acqui- 
sition, processing and analysis of epidemiological and clinical projects. 

In regard to IMPLEMENTATION, a decision was made to avoid the time-con- 
suming, often inaccurate manual translation of data from source materials to punched 
cards by utilizing original documents of data entry which could be machine-processed. 
Optically scanned forms incorporating pre-coded marking positions which provide 
for numerous response possiblities were considered the most appropriate for data 
collection and management. 

Three projects have been undertaken so far. The first (Automated Record Sys- 
tem for Computer Programming of Dental Treatment) encompassed the development 
of an oral examination record, which could be processed directly by an optical scan- 
ning machine, and a related computer program that generates treatment plans. The 
system is a compromise between sufficient and all-inclusive detail. Consequently, 
its use has been limited to general rather than special practice situations and has 
excluded emergency procedures and unusual surgery. The system is designed to 
cover 95 rather than 100% of cases. 

Over 500 computer-generated treatment plans have been compared with plans 
developed independently for the same patients by dentists. Agreement between treat- 
ment plans generated by computer and by dentists was 95%. Discrepancies observed 
were either the result of an inadequate data base or deviations by the dentists from 
criteria previously agreed upon. 

The second project (Automated Oral Examination Record for Epidemiological 
Studies) involved the development of a computer-aided system for the summarization 
and statistical analysis of results of dental examinations. The system has been de- 
signed for use in studies of dental caries prevalence and clinical trials. Computer 
programs have been prepared to edit data, provide summary information, make sur- 
face comparisons after successive examinations and perform statistical analyses. 

The third project (Computer Assisted System for Nutritional Analysis in Epi- 
demiological Studies) was designed to develop a computer-aided system for the sum- 
marization of findings of interviews concerning nutritional intakes. Utilization of 
food value tables, calculation of individual nutrients and other operations are done 
by computer. The system is useful in dietary research projects and in preventive 
dentistry. 

The COMPUTER (HARDWARE) SYSTEM consisted of the IBM 360/50. FOR- 
TRAN IV was the PROGRAMMING LANGUAGE used. In this project, pre-coded 
optically scannned sheets served as SPECIAL RECORD FORMATS. 

The STAFF for these projects consisted of the principal investigator plus con- 
sultants. Establishment of a separate computing center was avoided. Costs and per- 
sonnel pioLlems were minimized by renting use of hardware and purchasing services. 

FUTURE PLANS call for development of new methods to meet demands of 
increased data acquisition, as well as for greater use of the systems by students. 

Robert L. Glass, D.M.D., Dr. P.H. 

Associate Professor, Dentistry 
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Project Abstract: MEDICAL COLLEGE OF GEORGIA 

Computer Use at the School of Dentistry 

The PROJECT GOAL is to inaugurate a program of computer use 
unique to the School of Dentistry. At present, computer use by the 
School of Dentistry is limited to programs of general application to 
the institution: budget control, warehouse inventory control and a file 
on applicant information and admissions decisions. Faculty members 
have made use of the facilities on an individual basis. 

Projects for potential IMPLEMENTATION include a system in- 
corporating records for clinical instructors, students, and patients; a 
system of preventive maintenance, associated with the class scheduling 
system; a system of inventory control, monitoring obsolescence and 
reor ering of materials; and, perhaps most exciting, a system capable 
of simulating the effects of policy changes. 

The COMPUTER (HARDWARE) SYSTEM consists of the IBM 
360/30, located at the Medical College of Georgia Computer Center. 

The STAFF of the Computer Center includes a director, assistant 
director, manager of quality control, supervisor of programmers, sys- 
tems analysts, programmers, operators and clerical personnel. 

P. Kenneth Morse, Ph.D, 

Educational Psychologist 
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Project Abstract: NEW JERSEY COLLEGE OF MEDICINE 
AND DENTISTRY 
Computer Applications Planned 

The PROJECT GOAL is development of an operational computer 
system for use in connection with the new dental school now under 
construction. At present, the College of Dentistry does not have such a 
computer network, although investigation of candidate systems is under- 
way. 

It is hoped that the programs developed will have application in 

the following areas: 9 

L Matching complexity of patients’ dental treatment with students 
technical and academic capabilities. 

2. Monitoring of consistency and continuity of treatment in rela- 
tion to established treatment plan. 

3. Storage and print-out of students’ performance (grading). 

4. Student attendance in terms of appointments in each clinical 
discipline. 

5. Billing statements for completed dental treatment. 

6. Admissions procedures programmed to assist in the screening 
of applicants. 

7. Programmed instruction in clinical and basic science courses 
with an emphasis on correlating the two. 

8. Research data analysis to establish statistical significance of 
results. 

Ian C. Bennett, D.D.S. LeRoy A. Parker, Jn, D.D.S. 

D ean Assistant Dean, Clinical Affairs 
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Project Abstract: TUFTS UNIVERSITY 

SCASER: Student Clinic Assignment Schedule 
Evaluation and Report Program at the School 
of Dental Medicine 

SCASER is a computer program designed to aid in the preparation and evalua- 
tion of yearly student assignment schedules for a dental school and, once these are 
determined, to prepare suitable output reports to convey this scheduling information 
to the appropriate areas. The program generates any or all of six output reports 
on request. 

Output Report No* 1 is an Individual Student Assignment Schedule showing 
daily assignments, morning and afternoon, in the time period scheduled. In present 
form, up to forty weeks can be displayed on one page of computer output. 

Report No. 2, the Weekly Student Locator, is a guide for locating a student 
during a school day and indicates the assignments for all students in a given class 
on a weekly basis. 

Reports No. 3 and No. 4 list the students who are assigned to a given depart- 
ment every morning and afternoon for every week in the scheduling period. Student 
identification numbers (if any) are used in No. 3 and actual names in No. 4. 

Report No. 5 is a Class Size Print-out, prepared for each' day of the time 
period being scheduled, that indicates the utilization of facilities, i.e,, the number of 
students assigned to each teaching area on a morning/afternoon basis. 

Report No. 6 lists the number times each student is assigned to a given 
teaching area in a given scheduling period, and is used to make certain that the 
scheduling procedure assigns students the requisite number of times to each teaching 
area. 

SCASER has been found useful not only in handling actual scheduling tasks 
but also as a tool for analyzing the effects of proposed alternative schedule changes. 

The PROGRAMMING LANGUAGE is FORTRAN II, except for two short sub- 
routines written in assembly language. The COMPUTER (HARDWARE) SYSTEM 
currently includes the IBM 7094 and the Control Data Corporation 6600. 

With respect to data processing VOLUME, a schedule for 150 students for 40 
weeks requires 150 x 40 x 10, or 60,000 individual student assignments. Approxi- 
mately 2-3 man-days are required to prepare the input information in machine- 
readable form for a scheduling run of this size* The scheduling run itself requires 
about 40 minutes on the 7094 i and 18 minutes on the 6600. 

This work was supported in part by funds from a Public Health Service Special 
Research Fellowship fropi the Division of Dental Health, NIH. 

Mortimer L. Shakiui, D.D.S* 

Research Associate 
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Project Abstract: UNIVERSITY OF CALIFORNIA AT 
LOS ANGELES 

Computer Program in Clinical Management at 
the School of Dentistry 

The PROJECT GOAL is to establish a scientific information sys- 
tem for intelligent decision making, patient assignment and data re- 
trieval. The latter is to include patient histories and student clinical 
achievements stored for purposes of evaluation and research. A further 
objective is to make as few demands as possible on students and staff 
while developing this system. 

IMPLEMENTATION has taken the form of a dental records com- 
puter system composed of six programs: edit, update, student status 
report, patient queue report, division report and patient status report. 

The edit program verifies cards and prepares them for the update 
program, which services a master file and generates work files for the 

four work programs. ...... 

The student status report program, produced weekly, is divided 

into four parts: 

1. Treatment information regarding assigned patients. 

2. List of treatments completed, whether assigned or not, 

3. Summary of clinical work in terms of requirements for grad- 
uation. 

4. Clinical performance record. 

The patient queue program processes the treatment plan record 
for unassigned patients and is used in connection with part three of 
the student status report. The division report program generates infor- 
mation on all student accomplishments within a division, including 
ranked lists of student performance on particular operations. Finally, 
the patient status report program prints out stored and updated data in 
response to requests for information on certain patients. 

The COMPUTER (HARDWARE) SYSTEM includes the IBM 
360/50 and IBM 360/91, plus peripheral devices located at the Hospit- 
al Data Processing Facility and the Health Sciences Computing Facility. 

PROGRAMMING LANGUAGES in use are COBOL, 360 assembly 
language (BAL), PL/1, and FORTRAN. 

The STAFF of the Dental Data Processing Division includes a 
programmer, a programmer trainee, a keypunch operator, and a veri- 
fier, in addition to two clerks in patient assignment and a secretary. 
FUTURE PLANS involve development of 

1. a special payment program to streamline financial mangement, 

2. a retrieval program for evaluation and research, and 

3. computer-assisted instruction (CAI) materials. 

Samuel Cheng, D.D.S. 

Assistant Clinic Director 
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Project Abstract: UNIVERSITY OF IOWA 

Flexible Scheduling System at the College of 
Dentistry 

PROJECT GOALS revolve around the development of a model of 
a flexible dental education program, i.e. , a departure from the tradition- 
al lockstep scheduling of the professional curriculum. 

Partial IMPLEMENTATION has occurred in the form of the de- 
velopment and adoption of a new curriculum by the faculty of the 
College of Dentistry. The curriculum focuses effort on a core of know- 
ledge. skills, and abilities which are common to all dental specialties 
and activities. 

A student will not be compelled to learn what only a small per- 
centage may ever have reason to apply. Interest in special areas may 
be pursued by means of elective courses and students wilt be encourag- 
ed to be innovative in what courses they take and the sequence in 
which they take them. An enrichment program has been developed to 
assist them in this regard. In a further change, comprehensive clinical 
experience— diagnosis, treatment planning, and actual treatment done 
by the same student— -has been made part of the curriculum. 

In such a potentially random, individualized curriculum, continuity, 
both horizontal and vertical, will be difficult to maintain. In addition, 
the comprehensive patient care system will require coordination of 
diagnosis, selection, and scheduling of patients with individualized 
scheduling of students and clinical staff. 

Development of a system which would 

1. permit "shuffling” of numerous educational variables and cur- 
ricular options, 

2. meet the needs of the students, 

3. make an effective use of instructional staff, and 

4. maximize use of facilities, 

can be accomplished by means of computer procedures. The proposed 
project will involve combined resources of the College of Dentistry, the 
Measurement Research Center, and University Computer Center. The 
following major steps are planned: 

1. Systems analysis of departmental scheduling needs. 

2. Simulation of flexible schedule. 

3. Analysis of clinical operations. 

4. Development of computer programs (master schedule). 

5. Design of supporting subsystems. 

The primary outcome of the proposed project will be a working 
system of flexible and pedagogically sound scheduling of facilities, fac- 
ulty, students, patients, subject matter, and course content. 

STAFF includes a project director, a project associate director, a 
systems analyst, a programmer, clerical personnel and consultants. 

Richard M. Jacobs, D.D.S., Ph.D. 

Associate Dean 



166 DENTAL SCHOOL OPERATIONS-PROJECT ABSTRACTS 



Project Abstract: UNIVERSITY OF KENTUCKY 

Computer Usage in a Dental Clinical Teaching 
Program at the College of Dentistry 



The PROJECT GOAL is to provide current and comprehensive information re- 
garding the work of students in the Clinical Teaching Program, as well as on treat- 
ment plans and patient progress. The clinic curriculum is based on the total patient 
care approach. 

IMPLEMENTATION of the first phase has been accomplished by means of 
data acquisition procedures and appropriate computer programs designed to generate 
summaries of clinical work done by students. Data regarding a designated number 
of distinct clinical procedures are recorded on preprinted data cards during the clini- 
cal period and subsequently processed. 

The results of screening examinations of patients are processed to assist the 
Coordinator of Patient Care in assigning a balanced variety of patients to a student. 
A program has been developed which periodically generates a Patient Assignment 
Report indicating patient assignments and date of first appointment. Once a patient 
has been seen and a diagnosis made, information regarding him is deleted from the 
Patient Assignment Report and added to the Tieatment Plan Report, which lists 
treatment plan, supervising department, and dates of completed procedures. 

Finally, in addition to these larger and more integrated systems, a number of 
separate programs have been written for analysis of thn charges sent to the Business 
Office, and of the cash receipts, for control of student equipment, and for patient 
recall purposes. 

The COMPUTER (HARDWARE) SYSTEM is located in the Computing Center 
and includes the IBM System 360/50 with 2314 disk unit. The PROGRAMMING 
LANGUAGE employed is FORTRAN IV. The 360 Utility Sort and Merge has been 
used extensively. 

VOLUME of data processing amounted to 50 hours of computer time in the 
last year of operation. This figure includes time spent on program development and 
debugging; keypunched input to the various programs totaled 45,000 cards. 

The STAFF consists of an applications programmer/data coordinator (half-time), 
applications programmer/ analyst (part-time) and a keypunch/key verifier operator 
(full-time). 

FUTURE PLANS call for development of more and varied programs and for 
the provision of on-line disk storage and the installation of remote terminals to 
reduce the number of reports that must be generated periodically. 

Richard C. Bartlett, D.M.D. Garth L. Olde, Ph.D. 

Coordinator, Patient Care Director, Medical Center Computer Services 
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Project Abstract: UNIVERSITY OF MARYLAND 

Computer Usage for Undergraduate Dental 
Teaching Programs 

PROJECT GOALS involve more efficient handling of data related 
to scheduling, grades, clinical performance, counseling, student and 
patient progress, professional academic performance, and evaluation 
and research, as well as development of direct access inputs for infor- 
mation retrieval and design of computer-assisted instruction (CAI) 
programs. 

IMPLEMENTATION of the first group of activities has been ac- 
complished to a large extent, especially in the areas of class scheduling, 
student scheduling, student performance, and patient assignment. Most 
of the programs are relatively simple ones designed to store and satis- 
factorily retrieve large amounts of data recorded on a daily basis. Re- 
search applications of the computer are currently being investigated. 

The COMPUTER (HARDWARE) SYSTEM is the IBM 360/44 
located at the Health Sciences Computer Center. FORTRAN is employ- 
ed as the PROGRAMMING LANGUAGE. 

FUTURE PLANS involve work on the second group of goals; in- 
cluding installation of remote terminals and development of CAI ma- 
terials. 

Charles E. Barr, D.D.S. 

Assistant Dean for Clinical Sciences 
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Project Abstract: UNIVERSITY OF NEBRASKA 



Planned Computer Usage at the College of 
Dentistry 



The PROJECT GOAL involves the development of needed computer- 
based systems. The College of Dentistry is not using a computer facility 
at present. 

IMPLEMENTATION of goals is expected to take the form of an 
information storage and retrieval system with the following capabilities: 

1. Comparison of patient progress with previously diagnosed pa- 
tient needs. 

2. Comparison of performance of students working under the case 
system (comprehensive oral health care) with that of students 
working under a "requirements” system. 

3. Matching of patient needs with student needs. 

4. Providing dental manpower and demographic information on 
the State of Nebraska. 

5. Processing research data. 



The University has a COMPUTER (HARDWARE) SYSTEM that 
includes an IBM 360/65. FORTRAN IV and COBOL are in use as 
PROGRAMMING LANGUAGES. 

Gordon C. Vidmar, D.D.S. 

Chairman, Preventive Dentistry and Community Health 
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Project Abstract: UNIVERSITY OF NORTH CAROLINA 

Operations Research in Clinical Dental Education 
at the School of Dentistry 

PROJECT GOALS are: 

1. To demonstrate the merits of linear programming techniques for the 0 pti- 
mum allocation, scheduling and utilization of teaching staff and physical facilities in 
clinical dental education. 

2. To demonstrate its practical application through the total patient care con- 

cept. 

3. To make data available to other schools. 

IMPLEMENTATION of the first phase began with an analysis of the clinical 
teaching situation and subsequent identification of significant factors regarding the 
effectiveness and efficiency of clinical instruction. b 

These factors were then simulated conceptually in a mathematical model under 
the assumption that a linear relationship exists among them. The model was fed 
into the computer and studied using linear programming techniques. Results consti- 
tuted an identification of the elements that represent the clinical situation. Low-cost, 
nondisruptive experiments in curriculum change can be conducted using this work- 
ing model. 

The second phase involved a practical application of the linear programming 
model in the implementation of a total patient care projot. The project was designed 
to Provide an additional data base for the model; acquaint students, faculty and 
staff with automatic data processing; and introduce a new curriculum based on the 
comprehensive care concept. 

Participant selection was accomplished by dividing the junior class into five 
academic levels and choosing one student at random from each level. Students were 
assigned patients whose cases would provide a balanced clinical experience equal to 
or greater than the usual clinical requirements in the various departments. A pro- 
gram designed to match students and patients is not yet operational. 

1 he students* progress has been monitored and recorded for eventual compari- 
son with the performance of the rest of the class which served as the control group. 
Several criteria of project evaluation include, on a tentative basis, the quality, quan- 
tity and speed of clinical procedures and scores on mock board examinations, as- 
sessed using computer-executed statistical techniques. 

Clifton E. Crandell, D.D.S. 

Associate Professor, Oral Diagnosis 
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STUDENT EVALUATION IN CLINICAL 
COURSE WORK 



Report of Discussion Group 



Paul Ehrlich, Chairman 
Dean W. Darby, Recorder 

In our sessions we first went around the table and asked 
each group member to tell us a little bit about the work going 
on in his school and what he would like to see implemented in 
the future. I would like to present this material to you in an 
outline form in the interest of time. 

First, we discussed systems in operation or near completion. 
We found it was impossible to stay purely in student evaluation 
because it is tied so closely to other aspects of administration. 
Under systems in operation, our first topic was matching stu- 
dent needs and abilities with patient needs — as tied to the total 
patient care concept. Several schools have clinical records sys- 
tems in operation or underway. In the area of student rating, 
one of the schools also has an examiner rating system using a 
method of adjusting grades according to the student’s previous 
grading history. Many pros and cons developed on that subject. 
We then moved into student and patient billing systems. From 
the administrative point of view, both of these must be taken 
into consideration because we are all involved, or soon will be, 
in third-party payments. Also mentioned were the recall systems 
that some schools have for periodontics, pedodontics, and pre- 
ventive dentistry; backup card systems in grading in case there 
are downtime problems on the computer; and systems for inven- 
tory control, class scheduling, patient screening, and delinquency 
reporting. 

As for systems planned or being developed, we mentioned 
complete patient information systems, matching student needs 
and ability with patient needs; computer-assisted instruction; and 
student pre-dental performance records for admission which 
might include, it was suggested, psychological testing aspects 
which most of us neglect. Also under development are billing 
systems, several methods of student ratings, special assignment 
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criteria, oral diagnostic systems, methods for treatment planning, 
inventory control and clinical utilization of reports. 

The second major area of discussion was common problems 
and needs. These included money, personnel problems, coopera- 
tion (people problems), and coordination among students, faculty, 
and computer centers. There was a consensus that the faculty 
provided most of the obstacles to getting the job done. The exist- 
ence of decentralized facilities is also a problem in many schools. 
Most frequently mentioned was the need to share selected pro- 
grams and systems and to reduce duplication. Solutions could 
take the form, of a system for interchange of information or a 
series of core or profile packages that could be adapted to local 
needs, e.g., profile packages for student rating, or for matching 
patient needs with student needs and ability. 

A crucial problem is ensuring the acceptance and utility of 
the final product. Specialists familiar with dental problems design 
reports that are more meaningful, usable, and readable, than 
those who are purely computer specialists. Those of us who have 
been in operation for some time realize that unless the material 
you submit to the faculty and staff is in a usable form or is 
needed, it is of little value. A means for feedback to those who 
provide the inputs is very important in order to get their coop- 
eration. This includes the faculty and the students. Many of the 
faculty involved had never seen one of the print-outs that were 
obtained from the input they provided. In many cases, it ap- 
peared that it is the little things that make a system successful — 
for instance, card size, ease of handling for an instructor, the 
attitude toward the system, and a personalized print-out. 

Now, among the things that the participants hoped to get 
from the conference, the two most frequently mentioned were 
knowledge of ongoing programs and help in the design and 
planning of programs and systems. Two significant proposals 
were made. The first was to form a committee of representatives 
of schools who currently use clinical evaluation and related pro- 
grams. This committee will attempt to form a foundation for a 
general clinical evaluation program which will have application 
to all schools. Second, I would like to invite all participants at 
this conference to become part of the information-exchange 
group that was formed at the North Carolina Conference on 
Comprehensive Care in Clinical Dental Education. All partici- 
pants will be contacted shortly by questionnaire. 
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Project Abstract: CASE WESTERN RESERVE UNIVERSITY 
Computer-Assisted Clinical Grading at the 
School of Dentistry 

The PROJECT GOAL is to locate and correct shortcomings in the 
operative dentistry teaching program and to give students remedial help 
in problem areas. The program varies somewhat in different depart- 
ments. 

A preprinted and pre-coded grade card is made out for each step 
of each operation. Students are graded in terms of criteria of technical 
competence, professional behavior and reasoning ability peculiar to each 
operation. All grade cards are collected daily and a computer card is 
punched for each one. Finally, the cards are read into the computer 
and the program generates a magnetic tape sorted by various character- 
istics, e.g., patient number, student’s class, clinical procedure, etc. 

A print-out displays the data, including an average grade for each 
category of the procedure. This type of measure provides a better indi- 
cation of areas of weakness than an average grade over all categories. 

The system has in fact expanded the grading base because student 
grades are based on a broader group of criteria. In addition, grading 
has become more accurate and instances of "blanket grading” have 
been exposed. 

The General Electric 225 computer comprises the COMPUTER 
(HARDWARE) SYSTEM. The PROGRAMMING LANGUAGE is VFAP, 
an assembly language for GE 200-series computers. 

The borrowed STAFF is composed of a keypuncher and two con- 
sultants. 

FUTURE PLANS involve employing a keypunch operator responsi- 
ble only to the School of Dentistry. 

John P. H. Clark, D.D.S. 

Professor, Pharmacology and Operative Dentistry 
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Project Abstract t EMORY UNIVERSITY 

Data Processing Systems at the School 
of Dentistry 

PROJECT GOALS are to aid the Director of the Diagnostic Clinic 
in maintaining control of patient acceptance and assignment, and to 
provide administration and faculty with a comprehensive record of 
clinical treatment rendered, allowing them to evaluate both quantitative- 
ly and qualitatively. 

IMPLEMENTATION involves two computer related programs now 
in use. The first is a student scheduling program which generates a 
master schedule for one academic year showing Junior and Senior 
assignments to the various departments in which block assignment of 
students is applicable, including Oral Medicine (Diagnosis and Radiol- 
ogy), Emergency Service, Dental Assistant Utilization, Oral Surgery, 
Hospital Training and Pedodontics, 

The. second program handles patient admittance; assignment; 
treatment progress, as well as student clinical accomplishment; patient 
load (treatment plans); and evaluation. 

The COMPUTER (HARDWARE) SYSTEM consists of the RCA 
Spectra 70/55 located at the Computer Center. COBOL is the PRO- 
GRAMMING LANGUAGE in use. 

A SPECIAL RECORD FORMAT, the Standard Register Company 
Source Records System, was selected for use on the project. The sys- 
tem employs “zip-card” sets and master plates, resembling credit cards, 
containing static data. “Zip-cards,” like computer cards, are puncheo 
but the former have fewer data spaces and reduce error. Supporting 
programs in accounting, grading, patient recall and other areas have 
been developed using this records management system. 

STAFF includes two programmers from the University Computer 
Center and two consultants from the Standard Register Company. 

FUTURE PLANS call for achieving maximum machine utilization 
for matching the needs of clinic patients with those of students. 

Charles L. Halstead, D.D.S. 

Chairman, Oral Medicine 
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Project Abstract: LOMA LINDA UNIVERSITY 

Evaluation of Student Clinical Achievement by 
Computer at the School of Dentistry 

The PROJECT GOAL is to record and evaluate student progress in 
periodontal work with the aid of a computer. 

Insofar as IMPLEMENTATION is concerned, a separate port-a- 
punch card is made up for each procedure. It is the student’s respon- 
sibility to punch the appropriate information, which includes student 
number, patient number and procedure completed, on each card and 
have it initialed by the instructor, who punches out his number and 
the appropriate grade. The student must avoid committing any of the 
numerous errors possible in card punching, 

A code of abbreviations suitable for entry on cards has been de- 
veloped for various procedures in periodontic practice, such as initial 
preparation, case reevaluation, completion of active therapy and recall- 
maintenance procedures. A credit system corresponding to these pro- 
cedures has also been devised. Extra credit may be given in especially 
difficult cases and/or individual procedures. 

Students receive two types of clinical progress report print-outs: a 
cumulative report of all work completed to date and a periodic report 
containing all work completed since the last print-out. 

Again, it is the student’s responsibility to see that these reports 
are accurate and complete. Errors or omissions may be rectified by 
submitting a computer card containing the exact information to be 
deleted or added to the clinical instructor involved with the original 
problem. 

FUTURE PLANS include computer assignment of patients, and 
computer- man aged instruction in the following courses: Oral Diagnosis, 
Dental Materials, Cancer Recognition and Diagnosis, Practice Manage- 
ment, and Periodontology. 

Edwin M. Collins, D.D.S 
Assistant Dean 



G. Conrad Hornbuckle, D.D.S. 
Assistant Professor, Oral Medicine 
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Project Abstract: MEDICAL UNIVERSITY OF SOUTH 
CAROLINA 

Computerized Records Program at the School of 
Dentistry 

The PROJECT GOAL is a modern, efficient records program at 
the School of Dentistry. 

To date, IMPLEMENTATION has been achieved in activities asso- 
ciated with student clinical grading and accomplishments, clinical fees 
collected, and patient registration information. 

. T The COMPUTER (HARDWARE) SYSTEM is composed of the 
IBM System 360/30 and peripheral equipment, located in the Data 
Processing Center. The Center has, in addition, an IBM 2780 Data 
Transmission Terminal connected with the Common Research Compu- 
ter Facility at Houston, Texas. 

FUTURE PLANS call for the following: 

1. A system for matching patient and student clinical requirements 
for assignment purposes. 

2. Recording of laboratory activities and associated grading. 

3. Recording of didactic activities and associated grading. 

4. Establishment of a clinical and microscopic pathology registry. 

Benjamin F. Lawson, D.D.S, 

Chairman, Oral Medicine/Periodontology 
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Project Abstract: STATE UNIVERSITY OF NEW YORK AT 
BUFFALO 



PROJECT GOALS include integration of the health care problems of the patient 
with 9 tudent clinical training requirements. This involves development of more 
administratively efficient systems for retrieval of patient records and scientific data 
and for generation of student progress reports. The clinical training program i9 con- 
sidered the center of the entire educational effort. 

In the initial stage of IMPLEMENTATION, the School of Dentistry has begun 
to accumulate an "information reservoir,” specific entry to which is made through 
specialized computer programs. 

A master file of student and patient information is being developed to contain 
9uch data as 

1. static information on patients (address, phone, etc.), 

2. treatment plan information, including priorities and degree of difficulty, 

3. patient assignment information, based on availability, treatment require- 
ments, urgency, etc., 

4. patient progress information to be compiled daily, 

5. student progress reports which concisely display progre99 in each department 
and point out deficiencies, 

6. billing information, and 

7. limited scheduling information. 

The COMPUTER (HARDWARE) SYSTEM is the CDC 6400. PROGRAM- 
MING LANGUAGES include FORTRAN IV and COBOL. 

Data processing VOLUMES are as follows: 500 to 700 cards keypunched per 
day; three to four card-to-tape runs per week; and three weekly and five monthly 
reports. 

STAFF consists of a program mer/analy9t, part-time programmer/analyst, key- 
punch operator and private consultants when available. 

SPECIAL PROBLEMS include improper initial preparation and technical trans- 
cription of resource documents, limiting the accuracy of the results. Data reporting 
ha 9 been irregular because of incomplete development of the master file sections. 
Lack of control of patient information has been attributed to programming language 
problems. 

It has been found that the nature of the data processing task is not completely 
compatible with available hardware. That is, the CDC 6400 is mathematically 
oriented, while the Dental School’s needs revolve around summarizing, comparing 
and reformatting their data. The computing center ha9 been of limited help, mainly 
because dental school problems are outside its main 9tream of activity. 

FUTURE PLANS call for eliminating the need for "hard copy” processing by 
turning to hardware with display capability. In fact, an entirely different hardware 
9y9tem i9 presently under consideration. 

Nehon L. Blackmore, D.D.S. 

Chairman, Patient Admissions and Records 



The Application of Computer Science to the 
Administration of the School of Dentistry 
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Project Abstract: TEMPLE UNIVERSITY 

Computer-Assisted Student/Patient Management 
Procedures at the School of Dentistry 

t The PROJECT GOAL is to provide information regarding students’ 
clinical progress,^ grades and grade averages to faculty and students in 
an automated, efficient manner. 

IMPLEMENTATION initially involved selection of criteria by each 
department. Student performance during clinical experience is compared 
with those criteria and recorded on preprinted computer input cards. 
The cards are then fed into a National Cash Register machine which 
transfers the data to magnetic tape. 

Tapes are sent to the National Cash Register Company’s main 
office in Baltimore for print-out when needed. For all practical pur- 
poses, the system is fully debugged. 

The National Cash Register Computer, located in Baltimore, and 
the. NCR card read-in mechanism make up the COMPUTER (HARD- 
WARE) SYSTEM. 

VOLUME of data processing has been estimated at approximately 
175 input cards daily. Transfer of data requires between sixty and 
ninety minutes for one operator—18% of the total man-hours used pre- 
viously in keeping student grade and status records. 

FUTURE PLANS are as follows: 

1. To maintain an accurate administrative control over 8,000 pa- 
tient records. 

2. To maintain current knowledge as to the progress of each pa- 
tient through the treatment plan. 

3. To provide an effective recall system. 

4. To allow storage and retrieval of information concerning the 
incidence of oral diseases. 

Kendrick Brookreson, D.D.S. William E. Crolius, D.D.S. 

Chairman, Oral Diagnosis Assistant Dean, Clinical Affairs 



384 



180 CLINICAL EVALUATION-PROJECT ABSTRACTS 

Project Abstract: UNIVERSITY OF MISSOURI-KANSAS CITY 
A Computer-Supported Dental Educational 
System 



PROJECT GOALS revolve around including information science and computer 
techniques in a revised dental curriculum and incorporating such instruction into an 
extensive program of computer applications in dental education. 

IMPLEMENTATION of these objectives has been called the " Computer Sup- 
ported Dental Education System, which encompasses: 

1. Instruction in information science and computer techniques. 

2. Computer-aided instruction. 

3. Computer applications in educational research. 

4. Traditional administrative applications in the mangement of clinical records 
including both performance of students and patient records. 

First year students will be required to either demonstrate competence or take 
the course offered in computer science. Practical experience will be gained in the 
use of the dental information system as associated with clinical activities. Advanced 
courses in computer science will be offered as well. 

At present, two major files are being compiled: a patient file, based on data 
from the oral diagnosis clinic, and a related student file, prepared by the clinical 
teachers. The patient file includes such information as assignment, treatment and 
recall, while the student file contains such data as clinical procedures, quality of 
performance and professional evaluation. Students are rated in terms of weightings, 
or measures of difficulty, given by the faculty to all clinical procedures. 

Summary reports prepared from the student performance file are generated 
monthly and at the end of each semester. A mean quality index and an experience 
index are computed for each student. A list of students, ranked on each measure, 
is generated as well. 

Project STAFF includes a director, a biostatistician, two programmers, a tech- 
nical information scientist, an information clerk, four keypunch operators and two 

computer operators. .... 

The previous card system was a SPECIAL PROBLEM in that it was cumber- 
some and it managed only a limited amount of information. 

FUTURE PLANS involve: . , , , 

1. Further development of computer-aided instruction, particularly through the 
acquisition and use of cathode ray tube terminals. 

2. A redesign of the present records management system, 

3. Utilization of the statistical advantages of a large enrollment for the purpose 
of educational research. 

In addition, a new hardware system, featuring high-speed, direct-access storage, 
is being selected. 



Homer C. Jamison, D.D.S., Dr. P.H. 

Professor, Preventive and Community Dentistry 
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Project Abstract: UNIVERSITY OF PENNSYLVANIA 

Computer Applications at the School of 
Dental Medicine 

PROJECT GOALS involve computerization of student evaluation, 
preclinical and clinical, and of educational administration in order to 
implement a program of Comprehensive Care. 

IMPLEMENTATION will involve programs designed to handle 
student billing, grade reporting, patient monitoring, student monitoring 
and clinical scheduling. 

The COMPUTER (HARDWARE) SYSTEM will consist of an 
IBM 360/40, with an IBM 360/75 as a backup unit. The PROGRAM- 
MING LANGUAGE will be U.S.A. Standard COBOL, with possible 
subroutines in PL/1. 

The Project STAFF is presently composed in large part of person- 
nel from a private corporation contracted to assist in programming. 

FUTURE PLANS call for a fully operational stal ls by September 
1970. 

Anthony A. Vito, D.D.S. 

Assistant Dean for Clinical Services 
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Project Abstract: UNIVERSITY OF SOUTHERN CALIFORNIA 
A Computerized Student Evaluation System 
at the School of Dentistry' 

The PROJECT GOAL is to allow student and faculty to follow 
closely the student’s qualitative and quantitative progress toward depart- 
mental and degree requirements. 

IMPLEMENTATION has taken the form of DSIRS: the Dental 
Student Information Retrieval System. Students are issued 25 preprinted 
input forms, known collectively as the Computer Input Form Book. 

The forms are an integral part of the system in that they serve 
both as the basis for data input and as a permanent record of the 
clinical accomplishments of the student. Arranged in groups of five, 
they consist of the "Start Sheet” (white), the "Completion Sheet” 
(yellow), "Cashier's Sheet" (green), "Patient’s Receipt” (blue) and the 
"Student’s Copy” (pink). 

The appropriate forms are completed by student and instructor in 
the course of the clinical sessions and are subsequently collected. The 
sheets arc verified and data is transferred to punched cards. 

Two monthly reports are generated from this data. The Student 
Progress Report includes a summary of student’s standing, work in 
progress, work completed within the last six weeks (time period arbi- 
trary) and other indicators. A mathematical operation has been devised 
whereby measures of qualitative and quantitative performance are com- 
bined in one overall indicator. The Department Status Report displays 
the progress of every sti* ’.ent in each department. 

Programs which generate and support the preceding reports include 
an edit program, update error programs, summary program, calculation 
program, student print program and a departmental print program. 

The COMPUTER (HARDWARE) SYSTEM is a Honeywell 400, 
located at the University’s Data Processing Center. 

VOLUME of data processing includes punching roughly 1500 cards 
per week. Generation of all six related programs requires a 5 V 2 hour 
cycle using 14 tapes. 

FUTURE PLANS involve the addition of didactic and laboratory 
courses to the existing computer program, and the automatic analysis 
of clinic scheduling and assignment of patients to the proper students. 

Paul Ehrlich, D.D.S. 

Assistant Dean, Clinical Affairs 
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Project Abstract: UNIVERSITY OF TORONTO 

Examiner-Adjusted Clinical Grades Analysis 

The PROJECT COAL is to provide assistance in academic and udministrativc 
use of the digital computer. 

One aspect of IMPLEMENTATION is the scoring and analysis of Dental Apti- 
tude Test results, as well as other multiple choice tests. Another facet is the estab- 
lishment of record files for Burlington Orthodontic Research Centre participants. 

Perhaps the newest phase of operation involves the current version of the 
clinical grades analysis program which now features adjustment of examiners* sub* 
mitted marks prior to analysis and processing. 

The clinical grades analysis program is based on department requirements rather 
than a concept of total patient care and is designed to give a qualitative and quanti- 
tative assessment of student progress for each department. Students* grades and class 
standing are known to be influenced to a greater or lesser degree by rater differ- 
ences. Thus, the program was developed in response to the volume- of clinical assess- 
ments, the subjectivity of clinical examiners and the effect of being graded on stu- 
dent attitudes. 

To alleviate these problems, the grading scale is adjusted according to a rater’s 
past marking practice, in relation to performance of ail examiners in his depart- 
ment, before computation of scores. Once the raw term marks have been determined, 
they must be rescaled to a mean of 65%, in accordance with faculty policy. 

Forms of output include class, individual, and clinical examiner’s summaries. 
The individual summary lists each student as ranked on the basis of examiner- 
adjusted clinical averages from highest to lowest, while the class summary outlines 
the examiner-adjusted class average and the standard deviation of this distribution. 
Department heads have found the clinical examiner’s summary very useful. 

The Faculty of Dentistry has its own COMPUTER (HARDWARE) SYSTEM, 
which includes the IBM 1130 system with the recent addition of a Calcomp plot- 
ter. An IBM 514 punch is used on the students’ mark-sense grading cards. AS- 
SEMBLER and FORTRAN IV are utilized as PROGRAMMING LANGUAGES. 

In connection with VOLUME of data processing, it is estimated that 100,000 
grades are analyzed annually for two classes of 50 dental hygienist and two classes 
of 125 dental students. Furthermore, over 200 clinical staff contribute the marks. 

A two-sided mark-sense card, a SPECIAL RECORD FORMAT, is used as a 
record card. 

FUTURE PLANS involve examination and modification of existing operations, 
and the computer-generated scheduling of facilities, timetables, etc. Inventqry control 
is another possible application. 

Donald W. Lewis, D.D.S. 

Associate Professor, 

Dental Public Health and Biometrics 



Gordon W. Thompson, D.D.S. 
Research Associate, Epidemiology 
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Project Abstract: WEST VIRGINIA UNIVERSITY 

Computer Compilation Systems for Clinical 
Dental Records at the School of Dentistry 



The PROJECT GOAL revolves around management of dental records. 

IMPLEMENTATION has taken the form of 

1. a yearly scheduling program, 

2. development of a standard appointment card for all departments, 

3. agreement on a uniform numerical and color coding of each department and 
on a student identity code, 

4. a program designed to generate summaries of evaluation of performance in 
oral diagnosis, and 

5. four other general forms of records: 

a. departmental appointment statistics, 

b. a table of completed operations, 

c. a listing of all operations in progress, and 

d. a list of all students in each class, ranked according to a computation 
of all of the clinical evaluations. 

The COMPUTER (HARDWARE.) SYSTEM, located at the Computer Center, 
had consisted of the IBM 7040 central processing unit, but has been replaced by 
the IBM 360/75. Eventually, the 360/75 will be moved to a new facility, near the 
Medical Center. The PROGRAMMING LANGUAGE being used is FORTRAN IV. 

Yearly VOLUME of data processing in the Operative Dentistry Department has 
been estimated at roughtly 12,000 clinical data cards, which are converted into 
about 24,000 punched cards. 

The School of Dentistry has no project STAFF of its own, but relies heavily 
on the staff of the Computer Center and on part-time personnel when available. 

SPECIAL PROBLEMS arose in attempting to determine accurate estimates of 
costs. The Computer Center had not established general guidelines regarding finan- 
cial responsiblity. Computer time needed for debugging was so uncertain that pro- 
jects had to be grant-supported. Obtaining the services of a programmer has been 
another difficulty. Operations are dependent upon quality data, i.e., legible entries. 

FUTURE PLANS call for: 

1. Determination of choice of equipment and its installation and staffing in a 
Medical Center facility. 

2. The training of departmental secretarial staff for monitoring, storage and 
transfer of data cards. 

3. The extension of use of computer appointment cards to all clinical depart- 
ments. 

4. Modification of existing programs to give a more comprehensive record to 
each student of his accomplishments. 

Long range plans will involve admissions, grading, accounting, patient/Btudent 
matching, and alumni records. 

Robert E. Sausen, D.D.S. 

Chairman, Operative Dentistry 



APPENDIX 



( 




2.90 



Appendix 

List of Participants 



ABRAMS, Alan M., M.S. 

Statistician, Epidemiology Branch, Dental Health Center, 14th Ave- 
nue and Lake Street, San Francisco 94118. Tel: (415) 752-2560. 

ARIET, Mario, Ph.D. 

Coord., Health Center Computer, University of Florida, J. Hillis 
Miller Health Center, Gainesville 32601. Tel: (904) 392-3608. 

Areas : Computer-Assisted Instruction 

Status: Planned 

BARNETT, James M., M.P.A, 

Executive Assistant Director, Dental Health Center, 14th Avenue 
and Lake Street, San Francisco 94118. Tel: (415) 752-2560. 

BARR, Charles E., D.D.S. 

Assoc, Dean, Assoc. Prof., Periodontics, University of Maryland, 
School of Dentistry, Baltimore 21201. Tel: (301) 955-7461. 

Areas: Student Evaluation (Clinical, Academic), Educational Ad- 

ministration 

Computer System: IBM 360/44; FORTRAN IV 
Status: 4 years; ongoing 

BARTLETT, Richard C., D.M.D. 

Assoc, Prof., Coord., Patient Care, University of Kentucky College 
of Dentistry, Lexington 40506. Tel: (606) 233-5000, ext. 5411. 

Areas: Educational Administration 

Computer System: IBM 360/65; FORTRAN 

Status: 3 years; ongoing 

BENNETT, Ian C., D.D.S, 

Dean, New Jersey College of Dentistry, Jersey City 07304. 

Tel: (201) 877-4633. 

Areas : Educational Administration 

Status : Planned 

BLACKMORE, Nelson L., D.D.S. 

Prof, and Chm., Patient Adm. and Records, State University of 
New York at Buffalo School of Dentistry, Buffalo 14214. 

Tel: (716) 831-5381. 
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Areas: Student Evaluation (Clinical), Educational Administration 

Computer System: CDC 6400; FORTRAN IV, COBOL 

Status: 4 years; ongoing 

BLACKWELL, Robert B. 

Communications Specialist, Communication Services, Dental Health 
Center, 14th Avenue and Lake Street, San Francisco 94118. 

Tel: (415) 752-2560. 

BRESKE, Thomas B. 

Systems Analyst, Office of the Director, Dental Health Center, 14th 
and Lake Street, San Francisco 94118. Tel: (415) 752-2560. 

BRODY, Harvey A., D.D.S. 

Asst. Prof., Oral Biology and Dentistry, University of California 
School of Dentistry, San Francisco 94122. Tel: (415) 666-2045. 

Areas: Computer-Assisted Instruction 

Computer System: IBM 360/50; PILOT 

Status: l’year; ongoing 

BROOKRESON, Kendrick, D.D.S. 

Prof, and Chm., Oral Diagnosis, Temple University School of 
Dentistry, Philadelphia 19046. Tel: (215) 229-8500, ext. 352. 

Areas: Student Evaluation (Clinical), Educational Administration 

Computer System: NCR 

Status: 2 years; ongoing 

CAMPBELL, Edward M., D.D.S. 

Dir., Dental Manpower Development Center, Div. of Dental Health, 
Louisville 40202. Tel: (502) 582-5228. 

CARTER, Donald F. 

Design Engineer, Professional Education Branch, Dental Health 
Center, 14th Avenue and Lake Street, San Francisco 94118. 

Tel: (415) 752-2560. 

CHENG, Samuel, D.D.S. 

Asst. Clinic Dir., University of California School of Dentistry, Los 
Angeles 90024. Tel: (213) 825-5896. 

Areas: Admissions, Student Evaluation (Clinical), Educational Ad- 

ministration 

Computer Systems: IBM 36/50, IBM 360/91; FORTRAN, COBOL 
Status: 3 years; ongoing 

CLARK, John P. H., D.D.S. 

Prof., Pharmacology and Operative Dentistry. Chm., Operative 
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Dentistry* Case Western Reserve University School of Dentistrv, 
Cleveland 44106. Tel: (216) 368-3260. 

Areas: Student Evaluation (Clinical) 

Computer System: G.E. 225 

Status: 2 years; ongoing 

COLLINS, Edwin M., D.D.S. 

Asst. Dean. Prof., Diagnosis and Treatment Planning, Loma Linda 
University School of Dentistry. Loma Linda 92354. 

Tel: (714) 796-7414. 

Areas : Student Evaluation (Clinical) 

Computer System: FORTRAN 

Status: 3 years; ongoing 

COURY, Victor M., Ed.D. 

Prof, and Chm., Education and Counseling, University of Tennes- 
see College of Dentistry, Memphis 38103. 

Tel: (901) 527-6641, ext. 272. 

Areas : Student Evaluation (Academic), Computer-Assisted Instruc- 

tion 

Status: Planned 
CRANDELL, Clifton E., D.D.S. 

Dir., Dental Data Center, Assoc. Prof., Oral Diagnosis, University 
of North Carolina School of Dentistrv, Chapel Hill 27514. 

Tel: (919) 966-1161. 

Areas: Educational Administration 

Computer System: IBM 360/75; PL/1 

Status: 4 years; ongoing 

DARBY, Dean W., D.D.S., Ph.D. 

Dental Director, Professional Education Branch, Dental Health Cen- 
ter, 14th Avenue and Lake Street, San Francisco 94118. 

Tel: (415) 752-2560. 



DUNBAR, Charles R., M.S. 

Research Asst., Institute for Mathematical Studies in the Social 
Sciences, Stanford University, Stanford 94305. 

Tel: (415) 321-2300, ext. 2967. 

Areas: Computer-Assisted Instruction 

Computer Systems: IBM 1500, IBM 360/50, PDP-10; COURSE- 
WRITER, PILOT 
Status: 3 years; ongoing 
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DUROCHER, Roy T., D.D.S. 

Assoc. Dean and Dir. of Clinics, University of Pittsburgh Sqhool 
of Dental Medicine, Pittsburgh 15213. 

Tel: (412) 683-8400, ext. 608. 

Areas: Educational Administration, Computer-Assisted Instruction 
Computer Systems : IBM 7090, IBM 360/50, IBM 360/20, 

PDP-4; MAD, PL/1, FORTRAN IV, ALGOL, 
COBOL 

Status: 10 years; ongoing 
EHRLICH, Paul, D.D.S. 

Asst. Dean, Clinical Affairs, Assoc. Prof., Fixed Prosthodontics, 
University of Southern California School of Dentistry, Los Angeles 
90007. Tel: (213) 746-2821. 

Areas: Student Evaluation (Clinical) 

Computer System: Honeywell 400; COBOL 

Status: 3 years; ongoing 

FRENZEL, Winston W., D.D.S. 

Regional Dental Consultant, Region IX, Div. of Dental Health, 
Federal Office Building, 50 Fulton Street, San Francisco 94102. 
Tel: (415) 556-4410. 

FRIEND, Jamesine, B.A. 

Research Assoc., Institute for Mathematical Studies in the Social 
Sciences, Stanford University, Stanford 94305. 

Tel: (415) 321-2300, ext. 2954. 

Areas: Computer-Assisted Instruction 

Computer Systems: IBM 1500, PDP-1, PDP-10; ASSEMBLY 
LANGUAGE, COURSEWRITER 
Status: 5 years; ongoing 

GASTON, Gerald, D.D.S. 

Project Supervisor, Computer-Assisted Instruction Study, Continu- 
ing Education for Health Professions, Ohio State Regional Medical 
Program, Columbus 43210. Tel: (614) 293-8127. 

Areas: Computer-Assisted Instruction 

Computer System: IBM 360/40; COURSEWRITER III 

Status: 2 years; ongoing 

GINLEY, Thomas J., Ph.D. 

Assoc. Sec., Council on Dental Education, American Dental Associ- 
ation, Chicago 60611. Tel: (312) 944-6730. 

Areas: Admissions, Student Evaluation (Academic), Aptitude Test- 
ing Research 
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Computer System: Honeywell 200; EASY CODER, COBOL 

Status: 4 years; ongoing 

GLASS, Robert L., D.M.D., Dr. P.H. 

Assoc. Prof.', Dentistry, Forsyth Dental Center, Harvard Dental 
School, Boston 02115. Tel: (617) 734-3300, ext. 2267, or (617) 
262-5200. 

Areas: Epidemiology, Clinical Research 
Computer System: IBM 360/50; FORTRAN IV 

Status: 10 years; ongoing 

GRUBB, Ralph E., M.S. 

Research Psychologist, Education Research, International Business 
Machines Corporation, Monterey and Cottle Roads, San Jose, Cali- 
fornia 95114. Tel: (408) 227-7100, ext. 6232. 

HALSTEAD, Charles L., D.D.S. 

Assoc. Prof., Pathology, Chm., Oral Medicine, Emory University 
School of Dentistry, Atlanta 30322. Tel: (404) 377-2411, 
ext. 6856. 

Areas: Admissions, Student Evaluation (Clinical, Academic), Ed- 
ucational Administration 

Computer System: RCA Spectra 70/55; FORTRAN, COBOL 

Status: 1 year; ongoing 

HANSEN, Robert G., D.D.S. 

Assoc. Dean, University of Oklahoma School of Dentistry, Okla- 
homa City 73104. Tel: (313) 763-2105. 

Areas: Educational Administration 

Status: Planned; previously associated with University of Michi- 
gan project 

JACOBS, Richard M., D.D.S., Ph.D. 

Assoc. Dean, University of Iowa College of Dentistry, Iowa City 
52240. Tel: (319) 353-5723. 

Areas: Educational Administration 
Computer System: IBM 360/65; FORTRAN IV 

Status: 2 years; ongoing 



JAMISON, Homer C., D.D.S., Dr. P.H. 

Prof., Preventive and Community Dentistry, University of 
Missouri -Kansas City School of Dentistry, Kansas City 64106. 
Tel: (816) 842-0645, ext. 40. 



Areas: Student Evaluation (Clinical) 
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Computer System: IBM 360/30; FORTRAN IV, RPG 

Status: 1 year; ongoing 

JOLLEY, William H., D.D.S. 

Acting Dean, University of Tennessee College of Dentistry, Mem- 
phis 38103. Tel: (901) 527-6641, ext. 231. 

Areas: Student Evaluation (Academic), Computer- Assisted Instruc- 

tion 

Status: Planned 
KAMP, Martin, M.D. 

Chief, Instructional Division, Office of Information Systems, Uni- 
versity of California Medical Center, San Francisco 94122. 

Tel: (415) 666-2860. 

Areas : Computer-Assisted Instruction 

Computer System: IBM 360/50; PILOT 
Status: 4 years; ongoing 

KILLIP, Devore E., D.D.S. 

Dir., Division of Education Resources, University of Iowa College 
of Dentistry, Iowa City 52240. Tel: (319) 353-4380. 

Areas: Computer-Assisted Instruction 

Computer Systems: IBM 1500, IBM 360/65; COURSEWRITER III 
Status: 3 years; ongoing 

KLAPPER, Clarence E., Ph.D. 

Prof., Anatomy, Chm., Admissions Committee, University of Ala- 
bama School of Dentistry, Birmingham 35233. 

Tel: (205) 934-4566. 

Areas: Admissions 

Computer System: IBM 7040; FORTRAN IV 

Status: 2 years; ongoing 

KREIT, Leonard H., Ph.D. 

Research Psychologist, Professional Education Branch, Dental 
Health Center, 14th Avenue and Lake Street, San Francisco 94118. 
Tel: (415) 752-2560. 

KROEGER, Donald C., Ph.D. 

Prof, and Chm., Pharmacology, University of Texas Dental 
Branch, Houston 77025. Tel: (713) 529-4871, ext. 341. 

Areas: Admissions, Student Evaluation (Clinical, Academic), Ed- 

ucational Administration 

Computer Systems: IBM 7094, IBM 360/50; FORTRAN IV, PL/1 
Status: 3 years; ongoing 
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KUNIHIRA, Shirou, Ph.D. 

Assoc. Prof., Psychology, Loma Linda University School of Den- 
tistry, Loma Linda 92354. Tel: (714) 796-7414, ext. 659. 

Areas: Student Evaluation (Academic), Computer-Assisted 

Instruction 

Computer System: FORTRAN 

Status : 3 years; ongoing 

LAWSON, Benjamin F., D.D.S. 

Prof., Oral Medicine/Periodontology, Medical University of South 
Carolina College of Dental Medicine, Charleston 29401. 

Tel: (803) 792-3907. 

Areas: Student Evaluation (Clinical), Clinic Operations 

Computer System: IBM 360/30 
Status: 1 year; ongoing 

LEWIS, Donald W., D.D.S. 

Assoc. Prof. ,* Dental Public Health and Biometrics, University of 
Toronto Faculty of Dentistry, Toronto, Canada. 

Tel: (416) 928-6224. 

Areas: Student Evaluation (Clinical) 

Computer System: IBM 1130; FORTRAN IV 
Status: 10 years; ongoing 

LEWIS, Lloyd A., Ph.D. 

Division of Education Resources, University of Iowa College of 
Dentistry, Iowa City 52240. Tel: (319) 353-4380. 

Areas: Computer- Assisted Instruction 
Computer Systems: IBM 1500, IBM 360/65 
Status: 3 years; ongoing 

LUCACCINI, Luigi F., Ph.D. 

Research Psychologist, Professional Education Branch, Dental 
Health Center, 14th Avenue and Lake Street, San Francisco 94118. 
Tel: (415) 752-2560. 

LUCAS, Robert J., D.M.D. 

Chief, Training Branch, Dental Health Center, 14th Avenue and 
Lake Street, San Francisco 94118. Tel: (415) 752-2560. 



LUEBKE, Raymond G., D.D.S. 

Assoc. Dean, Louisiana State University School of Dentistry, New 
Orleans 70119. Tel: (504) 945-6857. 



Areas: Educational Administration 
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Status: Planned; previously associated with University of Ken- 
tucky project 

MACINTYRE, Martin L., D.D.S. 

Education Specialist, Professional Education Branch, Dental Health 
Center, 14th Avenue and Lake Street, San Francisco 94118. 

Tel: (415) 752-2560. 

MACKENZIE, Richard S., D.D.S., Ph.D. 

Prof, and Dir., Office of Dental Education, College of Dentistry, 
Prof., Education, College of Education, University of Florida, 
Gainesville 32601. Tel: (904) 392-3373. 

Areas ; Educational Research 

Status: Planned; previously associated with University of Pitts- 
burgh project 

MORSE, P. Kenneth, Ph.D. 

Educational Psychologist, Medical College of Georgia, Augusta 
30902. Tel' (404) 724-7111, ext. 272. 

Areas: Admissions. Simulation 
Status: Planned 

OLDE, Garth L., Ph.D. 

Dir., Division of Computer Resources and Services, Assoc. Prof., 
Behavioral Science, University of Kentucky Medical Center, Lex- 
ington 40506. Tel: (606) 233-5000, ext. 5201. 

Areas: Educational Administration 

Computer System: IBM 360/50; FORTRAN 

Status: 3 years; ongoing 

PAQUIN, Nancy A., M.A. 

Systems Analyst, Resource Analysis Branch, Div. of Dental Health, 
9000 Rockville Pike, Bethesda 20014. Tel: (301) 496-1347. 

PARKER, Jr., LeRoy A., D.D.S. 

Assoc. Dean, Clinical Affairs, Prof., Oral Diagnosis and Radiology, 
New Jersey College of Dentistry, Jersey City 07304. 

Tel: (201) 877-4654. 

Areas: Educational Administration 

Status: Planned 

PODSHADLEY, Dale W., D.D.S. 

Chief, Professional Education Branch, Dental Health Center, 14th 
Avenue and Lake Street, San Francisco 94118. 

Tel: (415) 752-2560. 
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ROZEN, Richard D., D.D.S. 

Specialist, Oral Biology, University of California School of Den- 
tistry, San Francisco 94122. Tel: (415) 666-2045. 

Areas : Computer-Assisted Instruction 

Computer System: IBM 360/50; PILOT 

Status: 1 year; ongoing 

RYGE, Gunnar, D.D.S. 

Scientific Director, Dental Health Center, 14th Avenue and Lake 
Street, San Francisco 94118. Tel: (415) 752-2560. 

SAUSEN, Robert E., D.D.S. 

Prof., Operative Dentistry, West Virginia University School of Den- 
tistry, Morgantown 26506. Tel: (304) 293-2611. 

Areas: Student Evaluation (Clinical) 

Computer Systems: IBM 7040, IBM 360/75; FORTRAN IV 
Status: 3 years; ongoing 

SCHROEDER, Robert C., B.S.C. 

Marketing Representative, International Business Machines Corp- 
oration, 340 Market Street, San Francisco 94111. 

Tel: (415) 823-3543. 

SEIBERT, Warren F., Ph.D. 

Prof., Educational Resources, University of Michigan School of 
Dentistry, Ann Arbor 48104. Tel: (313) 763-3337. 

Areas: Computer-Assisted Instruction 

Computer System: IBM 1130 

Status: 3 years; ongoing 

SHAKUN, Mortimer L., D.D.S. 

Research Assoc., Social Dentistry, Tufts University School of 
Dental Medicine, Boston 11374. Tel: (212.) 896-5064. 

Areas: Simulation, Operations Research, Automated X-Ray 

Analysis 

Computer System: IBM 7094 

Status: 2 years; ongoing 

SHROCK, John G., Ed.D. 

Research Psychologist, Training Branch, Dental Health Center, 
14th Avenue and Lake Street, San Francisco 94118. 

Tel: (415) 752-2560. 

SKOLNICK, Malcolm H., Ph.D. 

Dir., Biomedical Communications, State University of New York 
at Stony Brook, Stony Brook 11790. Tel: (516) 246-7985. 
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Areas: Computer- Assisted Instruction 

Computer System ; IBM 1130; FORTRAN 
Status : 2 years; ongoing 

SMITH, Roulette W., M.S. 

Asst. Prof., Psychology and Education, University of California, 
Santa Barbara 93106. Tel: (805) 961-2028, 961-3674. 

Areas: Educational Administration, Computer-Assisted Instruc- 

tion, Psychology 

Computer Systems: PDP-1, PDP-8, PDP-10; FAIL, MACRO, 



STARKWEATHER, John A., Ph.D. 

Dir., Office of Information Systems, Prof., Psychiatry. University 
of California Medical Center, San Francisco 94122. 

Tel: (415) 666-2012. 

STOLUROW, Lawrence M., Ph.D. 

Dir., Computer-Aided Instruction Laboratory, Harvard University 
Graduate School of Education, Cambridge 02138. 

Tel: (617) 495-4282. 

STRACHAN, Donald S., D.D.S., Ph.D. 

Asst. Dean, University of Michigan School of Dentistry, Ann Ar- 
bor 48104. Tel: (313) 763-3316. 

Areas: Educational Administration 

Computer Systems: IBM 1130, IBM 360/67; FORTRAN IV 
Status: 3 years; ongoing 

STURWOLD, Virginia G., Ed.S. 

Education Specialist, Communication Services, Dental Health Cen- 
ter, 14th Avenue and Lake Street, San Francisco 94118. 

Tel: (415) 752-2560. 

SWEARINGEN, Roger V., A.B. 

Education Specialist, Training Branch, Dental Health Center, 14th 
Avenue and Lake Street, San Francisco 94118. 

Tel: (415) 752-2560. 

TEUSCHER, George W., D.D.S. 

Rep., National Library of Medicine, Editor, J. of Dent. Educ., 
Dean, Northwestern University Dental School, Chicago 60611. 

Tel: (312) 649-8333. 

TIRA, Daniel E., Ph.D. 

Instructor, Division of Dental Education, Computer-Assisted In- 



GOGOL, LISP, PASS, COURSEWRITER 



Status: 6 years; ongoing 
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struction Study, Ohio State Regional Medical Program, Columbus 
43210. Tel: (614) 293-8127. 

Areas: Computer-Assisted Instruction 

Computer System : IBM 360/40; COURSEWRITER III, VER- 
SION 2 < 

Status: 2 years; ongoing f 

TODD, Jr., Aaron G., M.S. 

Dir., Instructional Technology, University of Southern California 
School of Dentistry, Los Angeles 90007. Tel: (213) 746-2892. 

Areas: Student Evaluation (Clinical, Academic), Individual 
Instruction 
Status: Planned 

TUCKER, Norman E., D.D. 

Asst. Chief, Continuing Education Branch, Div. of Physician 
Manpower, National Institutes of Health, Bethesda 20014. 

Tel: (301) 496-6772. 

VIDMAR, Gordon C., D.D.S. 

Asst. Prof, and Chm., Preventive Dentistry and Community Health, 
University of Nebraska College of Dentistry, Lincoln 68503. 

Tel: (402) 472-3161, ext. 226. ; 

Areas: Educational Administration 

Status: Planned 

VITO, Anthony, A., D.D.S. 

Assoc. Dean, Clinical Services, University of Pennsylvania School 
of Dental Medicine, Philadelphia 19104. Tel: (215) 594-8975. 

Areas : Student Evaluation (Clinical), Educational Administration 

Computer System ; IBM 360/75; COBOL 
Status: 2 years; ongoing 

WEBER, John C., D.D.S. 

Asst. Prof., Anatomy, Cornell University Medical College, New 
York 10021. Tel: (212) 879-9000, ext. 8193. 

Areas: Computer-Assisted Instruction 

Computer System: IBM 1500; COURSEWRITER II 

Status: 2 years; ongoing 

WEISENFELDER, Harry, B.A. 

Asst. Secretary and Dir., Division of Educational Research, Amer- 
ican Association of Dental Schools, Chicago 60611. 

Tel: (312) 944-5878. 
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ZIMMERMAN , Stuart 0 . , Ph . D . 

Assoc. Prof, and Head, Bio-Mathematics, University of Texas 
M.D. Anderson Hospital and University of Texas Dental Branch, 
Houston 77025. Tel: (713) 526-5411, ext. 451. 

Areas : Simulation, Biostatistics 

Computer Systems: SDS Sigma 5, EAI 680, IBM 7094; FOR- 
TRAN, SYMBOL 

Status: Ongoing 
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Conference Staff 



Dale W. Podshadley, D.D.S. 
Luigi F. Lucaccini, Ph.D. 
Martin L. MacIntyre, D.D.S. 



Admissions and Academic Evaluation 
Chairman: Weisenfelder 

Members: Abrams 
Ginley 
Jolley 
Klapper 
Kreit 

(Recorder) 
Kroeger 
Lewis (L.) 

Seibert 

Shrock 

Stachan 

Tira 

Todd 



Dental School Operations and 
Educational Administration 

Chairman: Crandell 

Members: Ariet 

Barnett 

(Recorder) 

Barr 

Bartlett 

Bennett 

Cheng 

Glass 

Hansen 

Jacobs 

Luebke 

Morse 

Shakiin 

Vidmar 



Conference Chairman 

Program Chairman 

Local Arrangements Chairman 

Computer- Assisted Instruction 
Chairman: Mackenzie 

Members: Brody 
Carter 
Coury 
Dunbar 
Durocher 
Friend 
Kamp 
Killip 
Kunihara 
Lucas 
Paquin 
Rozen 
Skolnick 
Smith 
Sturwold 
(Recorder) 
Swearingen 
Weber 
Zimmerman 

Student Evaluation in 
Clinical Coursework 

Chairman: Ehrlich 

Members: Blackmore 
Brook res on 
Clark 
Collins 
Darby 

(Recorder) 
Halstead 
Jamison 
Lawson 
Lewis (D.) 

Olde 
Parker 
Sausen 
Vito 



DISCUSSION GROUPS 
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